Sep 09 21:01:24.018330 ip-10-0-140-73 systemd[1]: Starting Kubernetes Kubelet... Sep 09 21:01:24.442769 ip-10-0-140-73 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 21:01:24.442769 ip-10-0-140-73 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 09 21:01:24.442769 ip-10-0-140-73 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 21:01:24.442769 ip-10-0-140-73 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 09 21:01:24.442769 ip-10-0-140-73 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 21:01:24.443431 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.442781 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 09 21:01:24.446566 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446550 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:01:24.446566 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446565 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446571 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446576 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446580 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446583 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446586 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446589 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446592 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446594 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446597 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446605 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446608 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446611 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446613 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446616 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446618 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446621 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446624 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446626 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:01:24.446636 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446629 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446631 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446634 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446637 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446641 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446645 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446647 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446650 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446653 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446656 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446658 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446660 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446664 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446666 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446669 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446671 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446674 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446676 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446679 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446682 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:01:24.447102 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446685 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446687 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446690 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446692 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446695 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446697 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446700 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446702 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446705 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446707 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446710 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446712 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446715 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446717 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446721 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446724 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446726 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446729 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446732 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446734 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:01:24.447592 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446738 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446740 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446743 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446745 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446748 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446750 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446753 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446755 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446757 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446760 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446762 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446765 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446768 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446770 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446773 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446775 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446778 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446780 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446783 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:01:24.448096 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446786 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446788 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446791 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446794 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446799 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446803 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446807 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446811 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446814 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446817 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446820 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446823 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446825 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446828 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446831 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446834 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446837 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446839 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446842 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:01:24.448552 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.446844 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:01:24.450027 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450015 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:01:24.450027 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450027 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450031 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450034 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450051 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450057 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450061 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450064 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450067 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450070 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450073 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450075 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450078 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450080 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450083 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450085 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450088 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450091 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450093 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450096 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450099 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:01:24.450106 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450102 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450105 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450108 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450110 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450113 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450117 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450122 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450125 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450128 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450131 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450134 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450137 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450141 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450143 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450146 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450149 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450151 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450155 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:01:24.450606 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450158 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450162 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450165 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450167 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450170 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450172 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450175 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450177 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450180 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450183 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450185 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450188 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450190 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450193 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450195 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450199 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450201 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450204 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450207 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:01:24.451058 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450210 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450213 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450216 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450218 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450221 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450224 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450226 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450229 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450231 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450234 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450236 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450238 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450241 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450244 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450246 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450249 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450252 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450254 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450258 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450260 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:01:24.451535 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450263 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450266 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450269 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450272 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450274 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450276 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450279 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450281 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450285 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450287 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450290 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450292 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450295 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450297 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450300 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450303 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450305 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450308 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450311 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450313 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:01:24.452031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450316 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450691 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450695 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450698 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450701 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450704 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450707 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450710 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450712 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450715 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450717 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450721 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450725 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450728 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450731 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450734 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450737 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450739 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450742 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:01:24.452605 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450745 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450748 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450751 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450754 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450757 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450759 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450762 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450765 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450767 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450770 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450773 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450775 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450778 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450780 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450783 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450785 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450788 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450790 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450793 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450796 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:01:24.453086 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450799 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450801 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450804 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450808 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450811 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450814 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450817 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450820 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450822 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450824 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450827 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450830 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450832 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450835 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450837 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450840 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450842 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450845 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450847 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450850 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:01:24.453596 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450852 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450855 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450857 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450860 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450862 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450865 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450867 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450870 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450872 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450875 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450878 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450880 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450883 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450886 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450889 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450891 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450893 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450896 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450898 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450901 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:01:24.454091 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450904 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450908 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450911 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450914 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450916 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450920 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450922 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450925 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450927 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450930 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450933 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450935 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450938 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450940 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450943 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450945 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450948 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450951 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450953 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450956 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:01:24.454579 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.450959 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451050 2560 flags.go:64] FLAG: --address="0.0.0.0" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451082 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451091 2560 flags.go:64] FLAG: --anonymous-auth="true" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451095 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451100 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451103 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451108 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451112 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451115 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451118 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451122 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451125 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451129 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451132 2560 flags.go:64] FLAG: --cgroup-root="" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451135 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451138 2560 flags.go:64] FLAG: --client-ca-file="" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451140 2560 flags.go:64] FLAG: --cloud-provider="external" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451143 2560 flags.go:64] FLAG: --cluster-dns="[]" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451147 2560 flags.go:64] FLAG: --cluster-domain="" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451150 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451153 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451156 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451160 2560 flags.go:64] FLAG: --container-log-max-files="5" Sep 09 21:01:24.455113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451163 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451167 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451170 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451173 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451176 2560 flags.go:64] FLAG: --contention-profiling="false" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451179 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451181 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451185 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451188 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451192 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451195 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451198 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451201 2560 flags.go:64] FLAG: --enable-load-reader="false" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451204 2560 flags.go:64] FLAG: --enable-server="true" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451207 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451215 2560 flags.go:64] FLAG: --event-burst="100" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451218 2560 flags.go:64] FLAG: --event-qps="50" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451221 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451224 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451227 2560 flags.go:64] FLAG: --eviction-hard="" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451232 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451235 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451238 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451241 2560 flags.go:64] FLAG: --eviction-soft="" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451244 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 09 21:01:24.455707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451247 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451250 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451253 2560 flags.go:64] FLAG: --experimental-mounter-path="" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451256 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451259 2560 flags.go:64] FLAG: --fail-swap-on="true" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451262 2560 flags.go:64] FLAG: --feature-gates="" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451266 2560 flags.go:64] FLAG: --file-check-frequency="20s" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451269 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451273 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451276 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451282 2560 flags.go:64] FLAG: --healthz-port="10248" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451285 2560 flags.go:64] FLAG: --help="false" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451288 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-140-73.ec2.internal" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451291 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451294 2560 flags.go:64] FLAG: --http-check-frequency="20s" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451297 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451309 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451313 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451316 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451319 2560 flags.go:64] FLAG: --image-service-endpoint="" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451322 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451325 2560 flags.go:64] FLAG: --kube-api-burst="100" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451334 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451338 2560 flags.go:64] FLAG: --kube-api-qps="50" Sep 09 21:01:24.456365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451340 2560 flags.go:64] FLAG: --kube-reserved="" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451343 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451346 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451351 2560 flags.go:64] FLAG: --kubelet-cgroups="" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451354 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451357 2560 flags.go:64] FLAG: --lock-file="" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451359 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451362 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451365 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451371 2560 flags.go:64] FLAG: --log-json-split-stream="false" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451373 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451377 2560 flags.go:64] FLAG: --log-text-split-stream="false" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451383 2560 flags.go:64] FLAG: --logging-format="text" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451386 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451389 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451392 2560 flags.go:64] FLAG: --manifest-url="" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451395 2560 flags.go:64] FLAG: --manifest-url-header="" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451399 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451403 2560 flags.go:64] FLAG: --max-open-files="1000000" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451407 2560 flags.go:64] FLAG: --max-pods="110" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451410 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451413 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451416 2560 flags.go:64] FLAG: --memory-manager-policy="None" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451419 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451422 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 09 21:01:24.456953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451425 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451428 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451435 2560 flags.go:64] FLAG: --node-status-max-images="50" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451438 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451441 2560 flags.go:64] FLAG: --oom-score-adj="-999" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451444 2560 flags.go:64] FLAG: --pod-cidr="" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451448 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451453 2560 flags.go:64] FLAG: --pod-manifest-path="" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451456 2560 flags.go:64] FLAG: --pod-max-pids="-1" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451459 2560 flags.go:64] FLAG: --pods-per-core="0" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451462 2560 flags.go:64] FLAG: --port="10250" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451467 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451470 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05cf5f2125df69b1f" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451473 2560 flags.go:64] FLAG: --qos-reserved="" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451475 2560 flags.go:64] FLAG: --read-only-port="10255" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451478 2560 flags.go:64] FLAG: --register-node="true" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451481 2560 flags.go:64] FLAG: --register-with-taints="" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451485 2560 flags.go:64] FLAG: --registry-burst="10" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451488 2560 flags.go:64] FLAG: --registry-qps="5" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451490 2560 flags.go:64] FLAG: --reserved-cpus="" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451493 2560 flags.go:64] FLAG: --reserved-memory="" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451497 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451500 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451503 2560 flags.go:64] FLAG: --rotate-certificates="false" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451505 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 09 21:01:24.457577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451510 2560 flags.go:64] FLAG: --runonce="false" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451512 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451515 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451518 2560 flags.go:64] FLAG: --seccomp-default="false" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451521 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451524 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451527 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451530 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451533 2560 flags.go:64] FLAG: --storage-driver-password="root" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451535 2560 flags.go:64] FLAG: --storage-driver-secure="false" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451538 2560 flags.go:64] FLAG: --storage-driver-table="stats" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451541 2560 flags.go:64] FLAG: --storage-driver-user="root" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451545 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451548 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451551 2560 flags.go:64] FLAG: --system-cgroups="" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451554 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451559 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451562 2560 flags.go:64] FLAG: --tls-cert-file="" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451566 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451572 2560 flags.go:64] FLAG: --tls-min-version="" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451575 2560 flags.go:64] FLAG: --tls-private-key-file="" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451577 2560 flags.go:64] FLAG: --topology-manager-policy="none" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451581 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451583 2560 flags.go:64] FLAG: --topology-manager-scope="container" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451586 2560 flags.go:64] FLAG: --v="2" Sep 09 21:01:24.458198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451591 2560 flags.go:64] FLAG: --version="false" Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451595 2560 flags.go:64] FLAG: --vmodule="" Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451598 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.451602 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451720 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451724 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451727 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451732 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451734 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451737 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451740 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451742 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451745 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451747 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451750 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451752 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451755 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451758 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451761 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451763 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451765 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:01:24.458836 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451768 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451774 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451776 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451779 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451783 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451785 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451788 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451792 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451796 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451799 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451802 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451805 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451808 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451810 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451813 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451815 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451818 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451820 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451824 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451827 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:01:24.459361 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451829 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451832 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451834 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451837 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451840 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451842 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451845 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451847 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451850 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451852 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451855 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451857 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451860 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451863 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451871 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451874 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451878 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451880 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451883 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:01:24.459859 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451885 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451888 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451890 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451893 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451895 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451898 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451901 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451903 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451906 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451908 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451911 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451915 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451917 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451920 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451930 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451933 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451936 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451938 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451941 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451943 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:01:24.460334 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451946 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451948 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451951 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451953 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451956 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451958 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451961 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451964 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451971 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451976 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451979 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451982 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451984 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451987 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451990 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451992 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451994 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.451998 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.452001 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:01:24.460831 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.452004 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.452007 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.452010 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.452013 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.452631 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.459940 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.459960 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.460749 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.460758 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460829 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460835 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460838 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460842 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460845 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460849 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:01:24.461305 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460851 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460854 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460856 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460859 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460862 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460864 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460867 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460869 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460872 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460875 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460878 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460880 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460882 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460887 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460891 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460894 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460896 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460899 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460902 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:01:24.461866 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460904 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460907 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460909 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460912 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460922 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460925 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460928 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460931 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460933 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460936 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460939 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460941 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460944 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460946 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460949 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460952 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460954 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460957 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460959 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460962 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:01:24.462487 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460964 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460967 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460970 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460972 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460975 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460977 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460980 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460982 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460985 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460987 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460990 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460992 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460995 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.460999 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461002 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461005 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461009 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461019 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461022 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:01:24.463031 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461026 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461029 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461032 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461035 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461054 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461059 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461062 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461065 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461067 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461070 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461072 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461075 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461077 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461080 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461082 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461085 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461087 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461090 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461092 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461095 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:01:24.463557 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461097 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461099 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461102 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461105 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461107 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461109 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461112 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461114 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461117 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461120 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461123 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461132 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461135 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461137 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.461140 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:01:24.464052 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.461145 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462510 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462572 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462583 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462589 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462594 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462599 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462603 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462608 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462613 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462618 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462622 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462627 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462632 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462642 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462647 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462650 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462653 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462658 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462661 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:01:24.464426 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462665 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462668 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462671 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462674 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462678 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462690 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462799 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462805 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462809 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462812 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462815 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462818 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462821 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462824 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462828 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462831 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462833 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462836 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462839 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:01:24.464902 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462842 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462845 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462847 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462850 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462853 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462856 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462858 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462863 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462867 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462870 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462873 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462876 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462879 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462882 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462885 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462887 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462890 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462893 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462896 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462900 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:01:24.465374 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462902 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462905 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462907 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462910 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462913 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462915 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462918 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462921 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462925 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462929 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462932 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462935 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462937 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462940 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462942 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462945 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462948 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462950 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462953 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:01:24.465858 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462955 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462958 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462960 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462963 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462965 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462968 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462971 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462974 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462977 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462980 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462983 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462987 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462990 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462993 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462995 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.462998 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.463001 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.463003 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.463006 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:01:24.466428 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.463008 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:01:24.466898 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.463011 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:01:24.466898 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:24.463014 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:01:24.466898 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.463020 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 21:01:24.466898 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.463163 2560 server.go:962] "Client rotation is on, will bootstrap in background" Sep 09 21:01:24.466898 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.466577 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 09 21:01:24.467619 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.467607 2560 server.go:1019] "Starting client certificate rotation" Sep 09 21:01:24.467719 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.467703 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 21:01:24.467755 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.467739 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 21:01:24.469945 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.469932 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 21:01:24.471887 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.471869 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 21:01:24.482972 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.482953 2560 log.go:25] "Validated CRI v1 runtime API" Sep 09 21:01:24.487103 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.487088 2560 log.go:25] "Validated CRI v1 image API" Sep 09 21:01:24.489059 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.489028 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 09 21:01:24.493861 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.493843 2560 fs.go:135] Filesystem UUIDs: map[2ccbfbbf-263e-42d5-9888-573ad8f9b292:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8d0972f3-0130-4fc5-88d8-4b8d29cef3b4:/dev/nvme0n1p3] Sep 09 21:01:24.493916 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.493861 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 09 21:01:24.499685 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.499663 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 21:01:24.500632 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.500518 2560 manager.go:216] Machine: {Timestamp:2026-09-09 21:01:24.49909321 +0000 UTC m=+0.360615597 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096962 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26334fa741b7270cc35ede750bbc1d SystemUUID:ec26334f-a741-b727-0cc3-5ede750bbc1d BootID:95187a58-dc18-4aee-ae79-1e038fb01fb3 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b8:ed:ee:f3:a7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b8:ed:ee:f3:a7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:ed:4e:f2:34:e1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 09 21:01:24.500632 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.500629 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 09 21:01:24.500763 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.500717 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 09 21:01:24.503503 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.503479 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 09 21:01:24.504005 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.503507 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-73.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 09 21:01:24.504652 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.504630 2560 topology_manager.go:138] "Creating topology manager with none policy" Sep 09 21:01:24.504741 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.504657 2560 container_manager_linux.go:309] "Creating device plugin manager" Sep 09 21:01:24.504741 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.504668 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 21:01:24.506617 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.506420 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 21:01:24.507399 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.507387 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 09 21:01:24.510296 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.510284 2560 state_mem.go:36] "Initialized new in-memory state store" Sep 09 21:01:24.510416 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.510407 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 09 21:01:24.510446 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.510442 2560 kubelet.go:486] "Attempting to sync node with API server" Sep 09 21:01:24.510502 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.510451 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 09 21:01:24.510502 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.510462 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 09 21:01:24.510502 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.510471 2560 kubelet.go:398] "Adding apiserver pod source" Sep 09 21:01:24.510502 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.510480 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 09 21:01:24.510820 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.510809 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 21:01:24.510902 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.510828 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 21:01:24.512978 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.512960 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 09 21:01:24.515089 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.515076 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 09 21:01:24.515135 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.515106 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 09 21:01:24.516450 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.516437 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 09 21:01:24.516511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.516453 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 09 21:01:24.516511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.516459 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 09 21:01:24.516511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.516465 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 09 21:01:24.516511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.516473 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 09 21:01:24.516511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.516479 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 09 21:01:24.516511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.516492 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 09 21:01:24.516511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.516497 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 09 21:01:24.516511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.516503 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 09 21:01:24.516511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.516508 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 09 21:01:24.516511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.516516 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 09 21:01:24.516766 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.516523 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 09 21:01:24.517709 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.517696 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 09 21:01:24.517743 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.517711 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 09 21:01:24.521569 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.521555 2560 server.go:1268] "Started kubelet" Sep 09 21:01:24.521699 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.521648 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 09 21:01:24.521768 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.521725 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 09 21:01:24.522003 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.521986 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-73.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 09 21:01:24.522418 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.522398 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-73.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 09 21:01:24.522483 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.522403 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 09 21:01:24.522483 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.522402 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 09 21:01:24.522587 ip-10-0-140-73 systemd[1]: Started Kubernetes Kubelet. Sep 09 21:01:24.523597 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.523025 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 09 21:01:24.525060 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.524746 2560 server.go:310] "Adding debug handlers to kubelet server" Sep 09 21:01:24.526788 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.526733 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 09 21:01:24.527523 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.527504 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 09 21:01:24.528689 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.528673 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 09 21:01:24.528809 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.528786 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found" Sep 09 21:01:24.529358 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.529340 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 09 21:01:24.529358 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.529347 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 09 21:01:24.529358 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.529356 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 09 21:01:24.529534 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.529443 2560 reconstruct.go:97] "Volume reconstruction finished" Sep 09 21:01:24.529534 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.529453 2560 reconciler.go:29] "Reconciler: start to sync state" Sep 09 21:01:24.529939 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.529926 2560 factory.go:55] Registering systemd factory Sep 09 21:01:24.529970 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.529942 2560 factory.go:223] Registration of the systemd container factory successfully Sep 09 21:01:24.530141 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.529194 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-73.ec2.internal.18d3c2d213743728 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-73.ec2.internal,UID:ip-10-0-140-73.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-73.ec2.internal,},FirstTimestamp:2026-09-09 21:01:24.521522984 +0000 UTC m=+0.383045371,LastTimestamp:2026-09-09 21:01:24.521522984 +0000 UTC m=+0.383045371,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-73.ec2.internal,}" Sep 09 21:01:24.530205 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.530180 2560 factory.go:153] Registering CRI-O factory Sep 09 21:01:24.530205 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.530197 2560 factory.go:223] Registration of the crio container factory successfully Sep 09 21:01:24.530272 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.530250 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 09 21:01:24.530324 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.530280 2560 factory.go:103] Registering Raw factory Sep 09 21:01:24.530324 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.530296 2560 manager.go:1196] Started watching for new ooms in manager Sep 09 21:01:24.530652 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.530640 2560 manager.go:349] Starting recovery of all containers Sep 09 21:01:24.538485 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.538251 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-73.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 09 21:01:24.538485 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.538438 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 09 21:01:24.543841 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.543666 2560 manager.go:354] Recovery completed Sep 09 21:01:24.548529 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.548418 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:24.550914 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.550894 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:24.550980 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.550926 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:24.550980 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.550936 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:24.551566 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.551550 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 09 21:01:24.551566 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.551562 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:24.551566 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.551565 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 09 21:01:24.551730 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.551584 2560 state_mem.go:36] "Initialized new in-memory state store" Sep 09 21:01:24.552236 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.552222 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:24.552319 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.552252 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:24.552319 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.552267 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:24.553326 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.553255 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-73.ec2.internal.18d3c2d21534a88f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-73.ec2.internal,UID:ip-10-0-140-73.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-73.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-73.ec2.internal,},FirstTimestamp:2026-09-09 21:01:24.550912143 +0000 UTC m=+0.412434529,LastTimestamp:2026-09-09 21:01:24.550912143 +0000 UTC m=+0.412434529,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-73.ec2.internal,}" Sep 09 21:01:24.564008 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.563941 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-73.ec2.internal.18d3c2d21534f14a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-73.ec2.internal,UID:ip-10-0-140-73.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-73.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-73.ec2.internal,},FirstTimestamp:2026-09-09 21:01:24.550930762 +0000 UTC m=+0.412453148,LastTimestamp:2026-09-09 21:01:24.550930762 +0000 UTC m=+0.412453148,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-73.ec2.internal,}" Sep 09 21:01:24.576350 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.576280 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-73.ec2.internal.18d3c2d215351671 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-73.ec2.internal,UID:ip-10-0-140-73.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-73.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-73.ec2.internal,},FirstTimestamp:2026-09-09 21:01:24.550940273 +0000 UTC m=+0.412462659,LastTimestamp:2026-09-09 21:01:24.550940273 +0000 UTC m=+0.412462659,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-73.ec2.internal,}" Sep 09 21:01:24.579523 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.579478 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rpqqs" Sep 09 21:01:24.585002 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.584933 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-73.ec2.internal.18d3c2d21534a88f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-73.ec2.internal.18d3c2d21534a88f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-73.ec2.internal,UID:ip-10-0-140-73.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-73.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-73.ec2.internal,},FirstTimestamp:2026-09-09 21:01:24.550912143 +0000 UTC m=+0.412434529,LastTimestamp:2026-09-09 21:01:24.552235435 +0000 UTC m=+0.413758031,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-73.ec2.internal,}" Sep 09 21:01:24.588534 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.588515 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rpqqs" Sep 09 21:01:24.594519 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.594459 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-73.ec2.internal.18d3c2d21534f14a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-73.ec2.internal.18d3c2d21534f14a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-73.ec2.internal,UID:ip-10-0-140-73.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-73.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-73.ec2.internal,},FirstTimestamp:2026-09-09 21:01:24.550930762 +0000 UTC m=+0.412453148,LastTimestamp:2026-09-09 21:01:24.552259363 +0000 UTC m=+0.413781753,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-73.ec2.internal,}" Sep 09 21:01:24.604990 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.604925 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-73.ec2.internal.18d3c2d215351671\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-73.ec2.internal.18d3c2d215351671 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-73.ec2.internal,UID:ip-10-0-140-73.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-73.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-73.ec2.internal,},FirstTimestamp:2026-09-09 21:01:24.550940273 +0000 UTC m=+0.412462659,LastTimestamp:2026-09-09 21:01:24.552272975 +0000 UTC m=+0.413795363,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-73.ec2.internal,}" Sep 09 21:01:24.629057 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.629025 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found" Sep 09 21:01:24.695209 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.695121 2560 policy_none.go:49] "None policy: Start" Sep 09 21:01:24.695209 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.695159 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 09 21:01:24.695209 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.695175 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 09 21:01:24.696265 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.696252 2560 policy_none.go:47] "Start" Sep 09 21:01:24.729735 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.729713 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found" Sep 09 21:01:24.730674 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.730661 2560 manager.go:337] "Starting Device Plugin manager" Sep 09 21:01:24.730755 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.730701 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 09 21:01:24.730755 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.730712 2560 server.go:87] "Starting device plugin registration server" Sep 09 21:01:24.731139 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.731126 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 09 21:01:24.731203 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.731140 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 09 21:01:24.731271 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.731256 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 09 21:01:24.731362 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.731351 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 09 21:01:24.731408 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.731364 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 09 21:01:24.732029 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.732010 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 09 21:01:24.732157 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.732078 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-73.ec2.internal\" not found" Sep 09 21:01:24.746255 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.746229 2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-73.ec2.internal\" not found" node="ip-10-0-140-73.ec2.internal" Sep 09 21:01:24.772058 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.772010 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 09 21:01:24.773338 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.773313 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 09 21:01:24.773438 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.773372 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 09 21:01:24.773438 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.773401 2560 kubelet.go:2448] "Starting kubelet main sync loop" Sep 09 21:01:24.773533 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.773490 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 09 21:01:24.776587 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.776566 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:01:24.831670 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.831641 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:24.832623 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.832605 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:24.832728 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.832636 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:24.832728 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.832646 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:24.832728 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.832667 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-73.ec2.internal" Sep 09 21:01:24.844186 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.844162 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-73.ec2.internal" Sep 09 21:01:24.844186 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.844188 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-73.ec2.internal\": node \"ip-10-0-140-73.ec2.internal\" not found" Sep 09 21:01:24.873934 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.873900 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal"] Sep 09 21:01:24.874101 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.873976 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:24.874524 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.874506 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found" Sep 09 21:01:24.875309 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.875285 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:24.875413 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.875322 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:24.875413 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.875335 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:24.876786 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.876773 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:24.876905 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.876885 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal" Sep 09 21:01:24.876952 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.876914 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:24.877517 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.877503 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:24.877593 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.877531 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:24.877593 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.877541 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:24.877593 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.877507 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:24.877707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.877602 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:24.877707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.877616 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:24.879162 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.879148 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" Sep 09 21:01:24.879230 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.879175 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:24.879813 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.879795 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:24.879892 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.879826 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:24.879892 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.879836 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:24.899673 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.899642 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-73.ec2.internal\" not found" node="ip-10-0-140-73.ec2.internal" Sep 09 21:01:24.904004 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.903985 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-73.ec2.internal\" not found" node="ip-10-0-140-73.ec2.internal" Sep 09 21:01:24.930682 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.930650 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ea201ea53f48274b482ba7553442dfe-config\") pod \"kube-apiserver-proxy-ip-10-0-140-73.ec2.internal\" (UID: \"1ea201ea53f48274b482ba7553442dfe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal" Sep 09 21:01:24.930817 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.930689 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6157f6ca1f732b195484b20e7d38cdee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal\" (UID: \"6157f6ca1f732b195484b20e7d38cdee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" Sep 09 21:01:24.930817 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:24.930714 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6157f6ca1f732b195484b20e7d38cdee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal\" (UID: \"6157f6ca1f732b195484b20e7d38cdee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" Sep 09 21:01:24.974631 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:24.974553 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found" Sep 09 21:01:25.031384 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.031356 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6157f6ca1f732b195484b20e7d38cdee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal\" (UID: \"6157f6ca1f732b195484b20e7d38cdee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" Sep 09 21:01:25.031384 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.031387 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6157f6ca1f732b195484b20e7d38cdee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal\" (UID: \"6157f6ca1f732b195484b20e7d38cdee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" Sep 09 21:01:25.031600 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.031410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ea201ea53f48274b482ba7553442dfe-config\") pod \"kube-apiserver-proxy-ip-10-0-140-73.ec2.internal\" (UID: \"1ea201ea53f48274b482ba7553442dfe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal" Sep 09 21:01:25.031600 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.031442 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ea201ea53f48274b482ba7553442dfe-config\") pod \"kube-apiserver-proxy-ip-10-0-140-73.ec2.internal\" (UID: \"1ea201ea53f48274b482ba7553442dfe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal" Sep 09 21:01:25.031600 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.031475 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6157f6ca1f732b195484b20e7d38cdee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal\" (UID: \"6157f6ca1f732b195484b20e7d38cdee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" Sep 09 21:01:25.031600 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.031473 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6157f6ca1f732b195484b20e7d38cdee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal\" (UID: \"6157f6ca1f732b195484b20e7d38cdee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" Sep 09 21:01:25.075486 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:25.075449 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found" Sep 09 21:01:25.175947 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:25.175906 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found" Sep 09 21:01:25.202118 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.202095 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal" Sep 09 21:01:25.206930 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.206913 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" Sep 09 21:01:25.277100 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:25.276988 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found" Sep 09 21:01:25.379238 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:25.379185 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found" Sep 09 21:01:25.468452 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.468409 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 09 21:01:25.468949 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.468589 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 21:01:25.480372 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:25.480332 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found" Sep 09 21:01:25.580570 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:25.580482 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found" Sep 09 21:01:25.585484 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.585465 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:01:25.628955 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.628931 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal" Sep 09 21:01:25.645536 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.645510 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:01:25.646032 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.646015 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 21:01:25.647383 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.647367 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" Sep 09 21:01:25.648592 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.648578 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:01:25.683312 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.683294 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 21:01:25.697559 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.697524 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 20:56:25 +0000 UTC" deadline="2028-02-25 02:14:57.259509837 +0000 UTC" Sep 09 21:01:25.697614 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:25.697558 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12797h13m31.561956861s" Sep 09 21:01:26.270659 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:26.270611 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6157f6ca1f732b195484b20e7d38cdee.slice/crio-12bd8627c9ba044fe1877c868e7b6e1cb96a858e78d88221e843178968116d17 WatchSource:0}: Error finding container 12bd8627c9ba044fe1877c868e7b6e1cb96a858e78d88221e843178968116d17: Status 404 returned error can't find the container with id 12bd8627c9ba044fe1877c868e7b6e1cb96a858e78d88221e843178968116d17 Sep 09 21:01:26.270914 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:26.270893 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ea201ea53f48274b482ba7553442dfe.slice/crio-58661333c482f845740d457c66ac81817912d15d310f46a317a76cc850ad40a6 WatchSource:0}: Error finding container 58661333c482f845740d457c66ac81817912d15d310f46a317a76cc850ad40a6: Status 404 returned error can't find the container with id 58661333c482f845740d457c66ac81817912d15d310f46a317a76cc850ad40a6 Sep 09 21:01:26.275987 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.275968 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:01:26.448133 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.448032 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:01:26.513054 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.513019 2560 apiserver.go:52] "Watching apiserver" Sep 09 21:01:26.521856 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.521830 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 09 21:01:26.522260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.522235 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-xbvrp","openshift-ovn-kubernetes/ovnkube-node-nhw7b","kube-system/konnectivity-agent-472cd","kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r","openshift-cluster-node-tuning-operator/tuned-fm9z8","openshift-image-registry/node-ca-sdc89","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal","openshift-multus/multus-mgt9t","openshift-network-operator/iptables-alerter-fxqcd","kube-system/global-pull-secret-syncer-gj6c4","openshift-dns/node-resolver-jq8n4","openshift-multus/multus-additional-cni-plugins-hmbd7","openshift-multus/network-metrics-daemon-mxztj"] Sep 09 21:01:26.524682 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.524663 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fxqcd" Sep 09 21:01:26.525927 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.525908 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.527215 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.527199 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-472cd" Sep 09 21:01:26.528536 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.528516 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 09 21:01:26.528655 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.528541 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:26.528655 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:26.528600 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:26.530003 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.529986 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.530537 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.530518 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 09 21:01:26.530661 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.530601 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:01:26.530661 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.530582 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nkfc7\"" Sep 09 21:01:26.530836 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.530820 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 09 21:01:26.531298 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.531283 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.532498 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.532473 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 09 21:01:26.532587 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.532571 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.533650 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.533632 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 09 21:01:26.533747 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.533669 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 09 21:01:26.533944 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.533926 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sdc89" Sep 09 21:01:26.535659 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.535642 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.535907 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.535881 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 09 21:01:26.536515 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.536496 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-td852\"" Sep 09 21:01:26.536861 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.536846 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 09 21:01:26.536910 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.536872 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 09 21:01:26.536910 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.536875 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 09 21:01:26.537274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.537254 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:26.537656 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:26.537622 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:26.537959 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.537679 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 09 21:01:26.537959 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.537883 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mnj4k\"" Sep 09 21:01:26.538232 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.538212 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8tklf\"" Sep 09 21:01:26.538317 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.538297 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 09 21:01:26.538371 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.538354 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mkp7c\"" Sep 09 21:01:26.538442 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.538354 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:01:26.538550 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.538531 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 09 21:01:26.540197 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.539906 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 09 21:01:26.540680 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.540653 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96d8365d-dc91-4abb-a692-9ace9f2740d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.540789 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.540696 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-tuned\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.540789 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.540733 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96d8365d-dc91-4abb-a692-9ace9f2740d3-cnibin\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.540789 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.540766 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96d8365d-dc91-4abb-a692-9ace9f2740d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.540941 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.540799 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-kubernetes\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.540941 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.540820 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h54wt\" (UniqueName: \"kubernetes.io/projected/e1caf595-b108-4c86-bde5-d032dbb6e7d4-kube-api-access-h54wt\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.541042 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.540995 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96d8365d-dc91-4abb-a692-9ace9f2740d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.541103 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.541068 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1b87a69-3799-43a2-8e4e-deffdbe8ae09-konnectivity-ca\") pod \"konnectivity-agent-472cd\" (UID: \"e1b87a69-3799-43a2-8e4e-deffdbe8ae09\") " pod="kube-system/konnectivity-agent-472cd" Sep 09 21:01:26.541275 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.541139 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-sys-fs\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.541275 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.541149 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:26.541275 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.541172 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-sysconfig\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.541275 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.541203 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-systemd\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.541486 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.541235 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-run\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.541486 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:26.541285 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:26.541486 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.541320 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1c8a253b-db72-4c92-9f00-f1ebf4d22a68-iptables-alerter-script\") pod \"iptables-alerter-fxqcd\" (UID: \"1c8a253b-db72-4c92-9f00-f1ebf4d22a68\") " pod="openshift-network-operator/iptables-alerter-fxqcd" Sep 09 21:01:26.541486 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.541359 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c8a253b-db72-4c92-9f00-f1ebf4d22a68-host-slash\") pod \"iptables-alerter-fxqcd\" (UID: \"1c8a253b-db72-4c92-9f00-f1ebf4d22a68\") " pod="openshift-network-operator/iptables-alerter-fxqcd" Sep 09 21:01:26.541486 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.541396 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96d8365d-dc91-4abb-a692-9ace9f2740d3-system-cni-dir\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.541711 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.541483 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96d8365d-dc91-4abb-a692-9ace9f2740d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.541711 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.541603 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.542178 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-socket-dir\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.542251 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542195 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m55b9\" (UniqueName: \"kubernetes.io/projected/729a2561-4ff3-4315-b771-6c5faacc6482-kube-api-access-m55b9\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.542251 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542223 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-sys\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.542345 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542249 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lzp9\" (UniqueName: \"kubernetes.io/projected/96d8365d-dc91-4abb-a692-9ace9f2740d3-kube-api-access-4lzp9\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.542345 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542307 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1b87a69-3799-43a2-8e4e-deffdbe8ae09-agent-certs\") pod \"konnectivity-agent-472cd\" (UID: \"e1b87a69-3799-43a2-8e4e-deffdbe8ae09\") " pod="kube-system/konnectivity-agent-472cd" Sep 09 21:01:26.542441 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542399 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-etc-selinux\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.542441 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542429 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-sysctl-d\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.542441 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542434 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jq8n4" Sep 09 21:01:26.542572 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542453 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-host\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.542572 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542493 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpr7t\" (UniqueName: \"kubernetes.io/projected/1c8a253b-db72-4c92-9f00-f1ebf4d22a68-kube-api-access-tpr7t\") pod \"iptables-alerter-fxqcd\" (UID: \"1c8a253b-db72-4c92-9f00-f1ebf4d22a68\") " pod="openshift-network-operator/iptables-alerter-fxqcd" Sep 09 21:01:26.542572 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542520 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-registration-dir\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.542572 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542543 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-var-lib-kubelet\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.542572 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542567 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs\") pod \"network-metrics-daemon-mxztj\" (UID: \"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41\") " pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:26.542826 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542592 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-lib-modules\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.542826 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542615 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1caf595-b108-4c86-bde5-d032dbb6e7d4-tmp\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.542826 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542641 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tknrf\" (UniqueName: \"kubernetes.io/projected/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-kube-api-access-tknrf\") pod \"network-metrics-daemon-mxztj\" (UID: \"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41\") " pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:26.542826 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542664 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-device-dir\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.542826 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542730 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-modprobe-d\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.542826 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542776 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-sysctl-conf\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.542826 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542787 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 09 21:01:26.543129 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542853 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 09 21:01:26.543129 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542867 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 09 21:01:26.543129 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.542874 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 09 21:01:26.544652 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.544635 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 09 21:01:26.545021 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.544888 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mbmcs\"" Sep 09 21:01:26.545021 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.544907 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wr5v7\"" Sep 09 21:01:26.545021 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.544921 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 09 21:01:26.545021 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.544949 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 09 21:01:26.545021 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.544964 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 09 21:01:26.545021 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.544996 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 09 21:01:26.546000 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.545981 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rrfjq\"" Sep 09 21:01:26.546100 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.546031 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 09 21:01:26.550088 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.550072 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 09 21:01:26.550998 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.550980 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jnlkr\"" Sep 09 21:01:26.551877 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.551862 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 09 21:01:26.564868 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.564843 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 21:01:26.582401 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.582383 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7g9d5" Sep 09 21:01:26.595265 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.595241 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7g9d5" Sep 09 21:01:26.631109 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.631084 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 09 21:01:26.642922 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.642901 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-node-log\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.643019 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.642933 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96d8365d-dc91-4abb-a692-9ace9f2740d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.643019 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.642951 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-socket-dir\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.643019 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.642968 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m55b9\" (UniqueName: \"kubernetes.io/projected/729a2561-4ff3-4315-b771-6c5faacc6482-kube-api-access-m55b9\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.643019 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643007 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-var-lib-cni-bin\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.643289 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643107 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/54a81a73-ac90-4e0b-894e-4b14e0069156-multus-daemon-config\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.643289 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643168 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-sysctl-d\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.643289 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-host\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.643289 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643199 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-socket-dir\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.643289 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643255 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-host\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.643289 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643267 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-sysctl-d\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.643555 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643296 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:26.643555 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643316 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96d8365d-dc91-4abb-a692-9ace9f2740d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.643555 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643321 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71b2bab2-31a6-4b4b-9f09-44ffbfd210c6-hosts-file\") pod \"node-resolver-jq8n4\" (UID: \"71b2bab2-31a6-4b4b-9f09-44ffbfd210c6\") " pod="openshift-dns/node-resolver-jq8n4" Sep 09 21:01:26.643555 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643355 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/54a81a73-ac90-4e0b-894e-4b14e0069156-cni-binary-copy\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.643555 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643390 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpr7t\" (UniqueName: \"kubernetes.io/projected/1c8a253b-db72-4c92-9f00-f1ebf4d22a68-kube-api-access-tpr7t\") pod \"iptables-alerter-fxqcd\" (UID: \"1c8a253b-db72-4c92-9f00-f1ebf4d22a68\") " pod="openshift-network-operator/iptables-alerter-fxqcd" Sep 09 21:01:26.643555 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-registration-dir\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.643555 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643453 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-registration-dir\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.643555 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643530 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-var-lib-kubelet\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.643555 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643554 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2a3ad848-e2c1-4909-98d6-f432895c189b-ovn-node-metrics-cert\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643572 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71b2bab2-31a6-4b4b-9f09-44ffbfd210c6-tmp-dir\") pod \"node-resolver-jq8n4\" (UID: \"71b2bab2-31a6-4b4b-9f09-44ffbfd210c6\") " pod="openshift-dns/node-resolver-jq8n4" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643587 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-run-k8s-cni-cncf-io\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643604 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1caf595-b108-4c86-bde5-d032dbb6e7d4-tmp\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643628 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/253428b3-7b9b-474c-b3eb-f8b8f63b9671-kubelet-config\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643671 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-run-netns\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643671 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-var-lib-kubelet\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643714 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-etc-kubernetes\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643750 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tknrf\" (UniqueName: \"kubernetes.io/projected/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-kube-api-access-tknrf\") pod \"network-metrics-daemon-mxztj\" (UID: \"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41\") " pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643779 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-modprobe-d\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643902 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-modprobe-d\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643913 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-run-netns\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643957 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-cni-bin\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.643986 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-var-lib-kubelet\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644014 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-tuned\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644039 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-kubelet\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.644084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644064 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-etc-openvswitch\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644109 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-run-openvswitch\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644138 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-kubernetes\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644165 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54wt\" (UniqueName: \"kubernetes.io/projected/e1caf595-b108-4c86-bde5-d032dbb6e7d4-kube-api-access-h54wt\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644190 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-var-lib-openvswitch\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644220 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1b87a69-3799-43a2-8e4e-deffdbe8ae09-konnectivity-ca\") pod \"konnectivity-agent-472cd\" (UID: \"e1b87a69-3799-43a2-8e4e-deffdbe8ae09\") " pod="kube-system/konnectivity-agent-472cd" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644238 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-kubernetes\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644247 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-sys-fs\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644285 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-sysconfig\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644304 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-systemd\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644319 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-run\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644324 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-sys-fs\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644340 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-run-multus-certs\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644313 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644366 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-systemd\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644368 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1c8a253b-db72-4c92-9f00-f1ebf4d22a68-iptables-alerter-script\") pod \"iptables-alerter-fxqcd\" (UID: \"1c8a253b-db72-4c92-9f00-f1ebf4d22a68\") " pod="openshift-network-operator/iptables-alerter-fxqcd" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644378 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-run\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.644894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644408 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c8a253b-db72-4c92-9f00-f1ebf4d22a68-host-slash\") pod \"iptables-alerter-fxqcd\" (UID: \"1c8a253b-db72-4c92-9f00-f1ebf4d22a68\") " pod="openshift-network-operator/iptables-alerter-fxqcd" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644431 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96d8365d-dc91-4abb-a692-9ace9f2740d3-system-cni-dir\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644447 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644455 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-sysconfig\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644467 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-sys\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644505 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-run-systemd\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644513 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-sys\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644525 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96d8365d-dc91-4abb-a692-9ace9f2740d3-system-cni-dir\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644531 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-multus-cni-dir\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644567 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644588 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-cnibin\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644585 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c8a253b-db72-4c92-9f00-f1ebf4d22a68-host-slash\") pod \"iptables-alerter-fxqcd\" (UID: \"1c8a253b-db72-4c92-9f00-f1ebf4d22a68\") " pod="openshift-network-operator/iptables-alerter-fxqcd" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644622 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644662 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lzp9\" (UniqueName: \"kubernetes.io/projected/96d8365d-dc91-4abb-a692-9ace9f2740d3-kube-api-access-4lzp9\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644688 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1b87a69-3799-43a2-8e4e-deffdbe8ae09-agent-certs\") pod \"konnectivity-agent-472cd\" (UID: \"e1b87a69-3799-43a2-8e4e-deffdbe8ae09\") " pod="kube-system/konnectivity-agent-472cd" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644720 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-etc-selinux\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644751 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-log-socket\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.645563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644774 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2a3ad848-e2c1-4909-98d6-f432895c189b-ovnkube-config\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644797 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76b95cc0-0a15-4c20-bd29-463861438b51-host\") pod \"node-ca-sdc89\" (UID: \"76b95cc0-0a15-4c20-bd29-463861438b51\") " pod="openshift-image-registry/node-ca-sdc89" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644819 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fqmf\" (UniqueName: \"kubernetes.io/projected/76b95cc0-0a15-4c20-bd29-463861438b51-kube-api-access-8fqmf\") pod \"node-ca-sdc89\" (UID: \"76b95cc0-0a15-4c20-bd29-463861438b51\") " pod="openshift-image-registry/node-ca-sdc89" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644842 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-slash\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644857 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1b87a69-3799-43a2-8e4e-deffdbe8ae09-konnectivity-ca\") pod \"konnectivity-agent-472cd\" (UID: \"e1b87a69-3799-43a2-8e4e-deffdbe8ae09\") " pod="kube-system/konnectivity-agent-472cd" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644875 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2a3ad848-e2c1-4909-98d6-f432895c189b-env-overrides\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644896 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-multus-conf-dir\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644918 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs\") pod \"network-metrics-daemon-mxztj\" (UID: \"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41\") " pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644922 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-etc-selinux\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644929 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1c8a253b-db72-4c92-9f00-f1ebf4d22a68-iptables-alerter-script\") pod \"iptables-alerter-fxqcd\" (UID: \"1c8a253b-db72-4c92-9f00-f1ebf4d22a68\") " pod="openshift-network-operator/iptables-alerter-fxqcd" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644945 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-lib-modules\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.644984 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/253428b3-7b9b-474c-b3eb-f8b8f63b9671-dbus\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645011 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-device-dir\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645032 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-sysctl-conf\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:26.645040 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645049 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-lib-modules\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645151 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/729a2561-4ff3-4315-b771-6c5faacc6482-device-dir\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.646254 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:26.645187 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs podName:bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:27.145164856 +0000 UTC m=+3.006687230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs") pod "network-metrics-daemon-mxztj" (UID: "bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645204 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-sysctl-conf\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645695 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkpbf\" (UniqueName: \"kubernetes.io/projected/71b2bab2-31a6-4b4b-9f09-44ffbfd210c6-kube-api-access-hkpbf\") pod \"node-resolver-jq8n4\" (UID: \"71b2bab2-31a6-4b4b-9f09-44ffbfd210c6\") " pod="openshift-dns/node-resolver-jq8n4" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645732 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-hostroot\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645758 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/76b95cc0-0a15-4c20-bd29-463861438b51-serviceca\") pod \"node-ca-sdc89\" (UID: \"76b95cc0-0a15-4c20-bd29-463861438b51\") " pod="openshift-image-registry/node-ca-sdc89" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645788 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96d8365d-dc91-4abb-a692-9ace9f2740d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645812 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-systemd-units\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645836 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-run-ovn\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645861 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-run-ovn-kubernetes\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645886 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xscn\" (UniqueName: \"kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn\") pod \"network-check-target-xbvrp\" (UID: \"e717cc32-b9ce-46d2-b624-1391699be568\") " pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645930 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-system-cni-dir\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645974 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-var-lib-cni-multus\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.645999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wllvk\" (UniqueName: \"kubernetes.io/projected/54a81a73-ac90-4e0b-894e-4b14e0069156-kube-api-access-wllvk\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.646030 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96d8365d-dc91-4abb-a692-9ace9f2740d3-cnibin\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.646072 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96d8365d-dc91-4abb-a692-9ace9f2740d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.646085 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96d8365d-dc91-4abb-a692-9ace9f2740d3-cnibin\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.647047 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.646098 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2a3ad848-e2c1-4909-98d6-f432895c189b-ovnkube-script-lib\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.647544 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.646132 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2px9f\" (UniqueName: \"kubernetes.io/projected/2a3ad848-e2c1-4909-98d6-f432895c189b-kube-api-access-2px9f\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.647544 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.646161 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-multus-socket-dir-parent\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.647544 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.646188 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-cni-netd\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.647544 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.646237 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96d8365d-dc91-4abb-a692-9ace9f2740d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.647544 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.646421 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96d8365d-dc91-4abb-a692-9ace9f2740d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.647544 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.646637 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96d8365d-dc91-4abb-a692-9ace9f2740d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.647544 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.646784 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96d8365d-dc91-4abb-a692-9ace9f2740d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.647957 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.647939 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1caf595-b108-4c86-bde5-d032dbb6e7d4-tmp\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.648032 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.647977 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e1caf595-b108-4c86-bde5-d032dbb6e7d4-etc-tuned\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.648242 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.648224 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1b87a69-3799-43a2-8e4e-deffdbe8ae09-agent-certs\") pod \"konnectivity-agent-472cd\" (UID: \"e1b87a69-3799-43a2-8e4e-deffdbe8ae09\") " pod="kube-system/konnectivity-agent-472cd" Sep 09 21:01:26.665107 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.665083 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpr7t\" (UniqueName: \"kubernetes.io/projected/1c8a253b-db72-4c92-9f00-f1ebf4d22a68-kube-api-access-tpr7t\") pod \"iptables-alerter-fxqcd\" (UID: \"1c8a253b-db72-4c92-9f00-f1ebf4d22a68\") " pod="openshift-network-operator/iptables-alerter-fxqcd" Sep 09 21:01:26.667464 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.667448 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h54wt\" (UniqueName: \"kubernetes.io/projected/e1caf595-b108-4c86-bde5-d032dbb6e7d4-kube-api-access-h54wt\") pod \"tuned-fm9z8\" (UID: \"e1caf595-b108-4c86-bde5-d032dbb6e7d4\") " pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.669254 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.669232 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m55b9\" (UniqueName: \"kubernetes.io/projected/729a2561-4ff3-4315-b771-6c5faacc6482-kube-api-access-m55b9\") pod \"aws-ebs-csi-driver-node-k4r7r\" (UID: \"729a2561-4ff3-4315-b771-6c5faacc6482\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.669458 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.669438 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tknrf\" (UniqueName: \"kubernetes.io/projected/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-kube-api-access-tknrf\") pod \"network-metrics-daemon-mxztj\" (UID: \"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41\") " pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:26.675646 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.675625 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lzp9\" (UniqueName: \"kubernetes.io/projected/96d8365d-dc91-4abb-a692-9ace9f2740d3-kube-api-access-4lzp9\") pod \"multus-additional-cni-plugins-hmbd7\" (UID: \"96d8365d-dc91-4abb-a692-9ace9f2740d3\") " pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.747284 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747200 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-cni-netd\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.747284 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747234 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-node-log\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.747284 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747254 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-var-lib-cni-bin\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.747284 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747273 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/54a81a73-ac90-4e0b-894e-4b14e0069156-multus-daemon-config\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.747612 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747301 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:26.747612 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747325 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71b2bab2-31a6-4b4b-9f09-44ffbfd210c6-hosts-file\") pod \"node-resolver-jq8n4\" (UID: \"71b2bab2-31a6-4b4b-9f09-44ffbfd210c6\") " pod="openshift-dns/node-resolver-jq8n4" Sep 09 21:01:26.747612 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747325 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-node-log\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.747612 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747328 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-cni-netd\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.747612 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747350 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/54a81a73-ac90-4e0b-894e-4b14e0069156-cni-binary-copy\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.747612 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747333 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-var-lib-cni-bin\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.747612 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747379 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2a3ad848-e2c1-4909-98d6-f432895c189b-ovn-node-metrics-cert\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.747612 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:26.747432 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:26.747612 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747461 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71b2bab2-31a6-4b4b-9f09-44ffbfd210c6-tmp-dir\") pod \"node-resolver-jq8n4\" (UID: \"71b2bab2-31a6-4b4b-9f09-44ffbfd210c6\") " pod="openshift-dns/node-resolver-jq8n4" Sep 09 21:01:26.747612 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747498 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-run-k8s-cni-cncf-io\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.747612 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747557 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71b2bab2-31a6-4b4b-9f09-44ffbfd210c6-hosts-file\") pod \"node-resolver-jq8n4\" (UID: \"71b2bab2-31a6-4b4b-9f09-44ffbfd210c6\") " pod="openshift-dns/node-resolver-jq8n4" Sep 09 21:01:26.747612 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747586 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-run-k8s-cni-cncf-io\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.747612 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:26.747617 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret podName:253428b3-7b9b-474c-b3eb-f8b8f63b9671 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:27.247594287 +0000 UTC m=+3.109116672 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret") pod "global-pull-secret-syncer-gj6c4" (UID: "253428b3-7b9b-474c-b3eb-f8b8f63b9671") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747642 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/253428b3-7b9b-474c-b3eb-f8b8f63b9671-kubelet-config\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747673 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-run-netns\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747699 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-etc-kubernetes\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747709 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/253428b3-7b9b-474c-b3eb-f8b8f63b9671-kubelet-config\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747727 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-run-netns\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747759 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-run-netns\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747776 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-cni-bin\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747801 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-run-netns\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747814 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-cni-bin\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747826 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-var-lib-kubelet\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747778 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-etc-kubernetes\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747856 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-kubelet\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747872 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-var-lib-kubelet\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747882 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-etc-openvswitch\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747909 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-kubelet\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747909 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-run-openvswitch\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747944 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-etc-openvswitch\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.748274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747951 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-run-openvswitch\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747956 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-var-lib-openvswitch\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747991 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-var-lib-openvswitch\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.747992 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-run-multus-certs\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748023 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/54a81a73-ac90-4e0b-894e-4b14e0069156-cni-binary-copy\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748030 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/54a81a73-ac90-4e0b-894e-4b14e0069156-multus-daemon-config\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748023 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-run-multus-certs\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748035 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-run-systemd\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748069 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71b2bab2-31a6-4b4b-9f09-44ffbfd210c6-tmp-dir\") pod \"node-resolver-jq8n4\" (UID: \"71b2bab2-31a6-4b4b-9f09-44ffbfd210c6\") " pod="openshift-dns/node-resolver-jq8n4" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748081 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-run-systemd\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748081 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-multus-cni-dir\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748146 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-cnibin\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748154 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-multus-cni-dir\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748178 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748207 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-cnibin\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748210 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-log-socket\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748241 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2a3ad848-e2c1-4909-98d6-f432895c189b-ovnkube-config\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748251 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748256 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76b95cc0-0a15-4c20-bd29-463861438b51-host\") pod \"node-ca-sdc89\" (UID: \"76b95cc0-0a15-4c20-bd29-463861438b51\") " pod="openshift-image-registry/node-ca-sdc89" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748283 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76b95cc0-0a15-4c20-bd29-463861438b51-host\") pod \"node-ca-sdc89\" (UID: \"76b95cc0-0a15-4c20-bd29-463861438b51\") " pod="openshift-image-registry/node-ca-sdc89" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748290 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fqmf\" (UniqueName: \"kubernetes.io/projected/76b95cc0-0a15-4c20-bd29-463861438b51-kube-api-access-8fqmf\") pod \"node-ca-sdc89\" (UID: \"76b95cc0-0a15-4c20-bd29-463861438b51\") " pod="openshift-image-registry/node-ca-sdc89" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748315 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-slash\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748318 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-log-socket\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748731 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2a3ad848-e2c1-4909-98d6-f432895c189b-env-overrides\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748780 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-multus-conf-dir\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748838 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/253428b3-7b9b-474c-b3eb-f8b8f63b9671-dbus\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748875 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkpbf\" (UniqueName: \"kubernetes.io/projected/71b2bab2-31a6-4b4b-9f09-44ffbfd210c6-kube-api-access-hkpbf\") pod \"node-resolver-jq8n4\" (UID: \"71b2bab2-31a6-4b4b-9f09-44ffbfd210c6\") " pod="openshift-dns/node-resolver-jq8n4" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748907 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-hostroot\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748942 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/76b95cc0-0a15-4c20-bd29-463861438b51-serviceca\") pod \"node-ca-sdc89\" (UID: \"76b95cc0-0a15-4c20-bd29-463861438b51\") " pod="openshift-image-registry/node-ca-sdc89" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.748969 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-systemd-units\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749002 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-run-ovn\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749072 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/253428b3-7b9b-474c-b3eb-f8b8f63b9671-dbus\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749112 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-run-ovn-kubernetes\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749170 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-multus-conf-dir\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749187 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-run-ovn-kubernetes\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749222 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-hostroot\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.749897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749238 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2a3ad848-e2c1-4909-98d6-f432895c189b-ovnkube-config\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749334 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-systemd-units\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749390 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-host-slash\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749434 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xscn\" (UniqueName: \"kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn\") pod \"network-check-target-xbvrp\" (UID: \"e717cc32-b9ce-46d2-b624-1391699be568\") " pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749477 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2a3ad848-e2c1-4909-98d6-f432895c189b-env-overrides\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749480 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-system-cni-dir\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749535 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-system-cni-dir\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749551 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2a3ad848-e2c1-4909-98d6-f432895c189b-run-ovn\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749584 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-var-lib-cni-multus\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749609 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wllvk\" (UniqueName: \"kubernetes.io/projected/54a81a73-ac90-4e0b-894e-4b14e0069156-kube-api-access-wllvk\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749643 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2a3ad848-e2c1-4909-98d6-f432895c189b-ovnkube-script-lib\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2px9f\" (UniqueName: \"kubernetes.io/projected/2a3ad848-e2c1-4909-98d6-f432895c189b-kube-api-access-2px9f\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749690 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-multus-socket-dir-parent\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749688 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-host-var-lib-cni-multus\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.749762 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/54a81a73-ac90-4e0b-894e-4b14e0069156-multus-socket-dir-parent\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.750455 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2a3ad848-e2c1-4909-98d6-f432895c189b-ovn-node-metrics-cert\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.750471 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/76b95cc0-0a15-4c20-bd29-463861438b51-serviceca\") pod \"node-ca-sdc89\" (UID: \"76b95cc0-0a15-4c20-bd29-463861438b51\") " pod="openshift-image-registry/node-ca-sdc89" Sep 09 21:01:26.750713 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.750515 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2a3ad848-e2c1-4909-98d6-f432895c189b-ovnkube-script-lib\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.766437 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:26.766408 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:01:26.766437 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:26.766435 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:01:26.766651 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:26.766449 2560 projected.go:196] Error preparing data for projected volume kube-api-access-5xscn for pod openshift-network-diagnostics/network-check-target-xbvrp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:26.766651 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:26.766508 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn podName:e717cc32-b9ce-46d2-b624-1391699be568 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:27.266492099 +0000 UTC m=+3.128014475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5xscn" (UniqueName: "kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn") pod "network-check-target-xbvrp" (UID: "e717cc32-b9ce-46d2-b624-1391699be568") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:26.768761 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.768728 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fqmf\" (UniqueName: \"kubernetes.io/projected/76b95cc0-0a15-4c20-bd29-463861438b51-kube-api-access-8fqmf\") pod \"node-ca-sdc89\" (UID: \"76b95cc0-0a15-4c20-bd29-463861438b51\") " pod="openshift-image-registry/node-ca-sdc89" Sep 09 21:01:26.768877 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.768825 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2px9f\" (UniqueName: \"kubernetes.io/projected/2a3ad848-e2c1-4909-98d6-f432895c189b-kube-api-access-2px9f\") pod \"ovnkube-node-nhw7b\" (UID: \"2a3ad848-e2c1-4909-98d6-f432895c189b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.770402 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.770382 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wllvk\" (UniqueName: \"kubernetes.io/projected/54a81a73-ac90-4e0b-894e-4b14e0069156-kube-api-access-wllvk\") pod \"multus-mgt9t\" (UID: \"54a81a73-ac90-4e0b-894e-4b14e0069156\") " pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.770498 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.770473 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkpbf\" (UniqueName: \"kubernetes.io/projected/71b2bab2-31a6-4b4b-9f09-44ffbfd210c6-kube-api-access-hkpbf\") pod \"node-resolver-jq8n4\" (UID: \"71b2bab2-31a6-4b4b-9f09-44ffbfd210c6\") " pod="openshift-dns/node-resolver-jq8n4" Sep 09 21:01:26.778715 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.778664 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal" event={"ID":"1ea201ea53f48274b482ba7553442dfe","Type":"ContainerStarted","Data":"58661333c482f845740d457c66ac81817912d15d310f46a317a76cc850ad40a6"} Sep 09 21:01:26.779830 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.779810 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" event={"ID":"6157f6ca1f732b195484b20e7d38cdee","Type":"ContainerStarted","Data":"12bd8627c9ba044fe1877c868e7b6e1cb96a858e78d88221e843178968116d17"} Sep 09 21:01:26.835271 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.835229 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fxqcd" Sep 09 21:01:26.844278 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.844256 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hmbd7" Sep 09 21:01:26.844510 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:26.844489 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c8a253b_db72_4c92_9f00_f1ebf4d22a68.slice/crio-72220c58ee1c0240dd41d4e8b68c056ea3eef2e9dee3736d0c4525ca77edff66 WatchSource:0}: Error finding container 72220c58ee1c0240dd41d4e8b68c056ea3eef2e9dee3736d0c4525ca77edff66: Status 404 returned error can't find the container with id 72220c58ee1c0240dd41d4e8b68c056ea3eef2e9dee3736d0c4525ca77edff66 Sep 09 21:01:26.851715 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.851696 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-472cd" Sep 09 21:01:26.851816 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:26.851772 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d8365d_dc91_4abb_a692_9ace9f2740d3.slice/crio-006cccb86d1e20b3813af76875e12fcafdf4aed950a4d1974856657e08d5fa9e WatchSource:0}: Error finding container 006cccb86d1e20b3813af76875e12fcafdf4aed950a4d1974856657e08d5fa9e: Status 404 returned error can't find the container with id 006cccb86d1e20b3813af76875e12fcafdf4aed950a4d1974856657e08d5fa9e Sep 09 21:01:26.857608 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.857581 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" Sep 09 21:01:26.859948 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:26.859921 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1b87a69_3799_43a2_8e4e_deffdbe8ae09.slice/crio-bf3fabe207dca0fdae03f00b2fc3116786b4db889b9030a12f57972eb76cfaae WatchSource:0}: Error finding container bf3fabe207dca0fdae03f00b2fc3116786b4db889b9030a12f57972eb76cfaae: Status 404 returned error can't find the container with id bf3fabe207dca0fdae03f00b2fc3116786b4db889b9030a12f57972eb76cfaae Sep 09 21:01:26.863024 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.862860 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" Sep 09 21:01:26.868405 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:26.868382 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod729a2561_4ff3_4315_b771_6c5faacc6482.slice/crio-5305338255d9d30650beb5aeeee4ebf8dce23541b38103ce7fe281348974f93d WatchSource:0}: Error finding container 5305338255d9d30650beb5aeeee4ebf8dce23541b38103ce7fe281348974f93d: Status 404 returned error can't find the container with id 5305338255d9d30650beb5aeeee4ebf8dce23541b38103ce7fe281348974f93d Sep 09 21:01:26.869061 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.869041 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mgt9t" Sep 09 21:01:26.871512 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:26.871478 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1caf595_b108_4c86_bde5_d032dbb6e7d4.slice/crio-73a100a05b37c4f9326a31dcb2df87f5a3d899f9cd7afcad249e634f677a91ee WatchSource:0}: Error finding container 73a100a05b37c4f9326a31dcb2df87f5a3d899f9cd7afcad249e634f677a91ee: Status 404 returned error can't find the container with id 73a100a05b37c4f9326a31dcb2df87f5a3d899f9cd7afcad249e634f677a91ee Sep 09 21:01:26.874176 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.874160 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sdc89" Sep 09 21:01:26.877240 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:26.877218 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a81a73_ac90_4e0b_894e_4b14e0069156.slice/crio-45a7e5f5c812e366d6daf1651f24af97d1457c7aceda28f3b71201ea83caa5c2 WatchSource:0}: Error finding container 45a7e5f5c812e366d6daf1651f24af97d1457c7aceda28f3b71201ea83caa5c2: Status 404 returned error can't find the container with id 45a7e5f5c812e366d6daf1651f24af97d1457c7aceda28f3b71201ea83caa5c2 Sep 09 21:01:26.879641 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.879619 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:26.882748 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:26.882730 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b95cc0_0a15_4c20_bd29_463861438b51.slice/crio-b34a85ebe35057e0186d35fcf7971e36a566380de89a65612e1e2a08835ac8e4 WatchSource:0}: Error finding container b34a85ebe35057e0186d35fcf7971e36a566380de89a65612e1e2a08835ac8e4: Status 404 returned error can't find the container with id b34a85ebe35057e0186d35fcf7971e36a566380de89a65612e1e2a08835ac8e4 Sep 09 21:01:26.884049 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:26.883998 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jq8n4" Sep 09 21:01:26.887946 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:26.887777 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a3ad848_e2c1_4909_98d6_f432895c189b.slice/crio-4cb3cd48110186d36cf2a35ed36548124c6287abd1a63cf5692d481ed8ec85c7 WatchSource:0}: Error finding container 4cb3cd48110186d36cf2a35ed36548124c6287abd1a63cf5692d481ed8ec85c7: Status 404 returned error can't find the container with id 4cb3cd48110186d36cf2a35ed36548124c6287abd1a63cf5692d481ed8ec85c7 Sep 09 21:01:26.892283 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:01:26.892265 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b2bab2_31a6_4b4b_9f09_44ffbfd210c6.slice/crio-ba7b9c4d454acc29c3acbc5ccd46863c399e9bbab3cf0ab79ca3e6246a526116 WatchSource:0}: Error finding container ba7b9c4d454acc29c3acbc5ccd46863c399e9bbab3cf0ab79ca3e6246a526116: Status 404 returned error can't find the container with id ba7b9c4d454acc29c3acbc5ccd46863c399e9bbab3cf0ab79ca3e6246a526116 Sep 09 21:01:27.153218 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.153138 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs\") pod \"network-metrics-daemon-mxztj\" (UID: \"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41\") " pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:27.153350 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:27.153296 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:27.153388 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:27.153368 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs podName:bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:28.153346866 +0000 UTC m=+4.014869256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs") pod "network-metrics-daemon-mxztj" (UID: "bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:27.254423 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.254387 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:27.254618 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:27.254527 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:27.254618 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:27.254598 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret podName:253428b3-7b9b-474c-b3eb-f8b8f63b9671 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:28.254579882 +0000 UTC m=+4.116102272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret") pod "global-pull-secret-syncer-gj6c4" (UID: "253428b3-7b9b-474c-b3eb-f8b8f63b9671") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:27.354839 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.354735 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xscn\" (UniqueName: \"kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn\") pod \"network-check-target-xbvrp\" (UID: \"e717cc32-b9ce-46d2-b624-1391699be568\") " pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:27.354983 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:27.354856 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:01:27.354983 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:27.354878 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:01:27.354983 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:27.354889 2560 projected.go:196] Error preparing data for projected volume kube-api-access-5xscn for pod openshift-network-diagnostics/network-check-target-xbvrp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:27.354983 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:27.354936 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn podName:e717cc32-b9ce-46d2-b624-1391699be568 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:28.354922465 +0000 UTC m=+4.216444840 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5xscn" (UniqueName: "kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn") pod "network-check-target-xbvrp" (UID: "e717cc32-b9ce-46d2-b624-1391699be568") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:27.598284 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.598201 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:56:27 +0000 UTC" deadline="2028-02-19 07:23:32.887099623 +0000 UTC" Sep 09 21:01:27.598284 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.598247 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12658h22m5.288858405s" Sep 09 21:01:27.773689 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.773653 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:27.773862 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:27.773800 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:27.788152 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.788093 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" event={"ID":"2a3ad848-e2c1-4909-98d6-f432895c189b","Type":"ContainerStarted","Data":"4cb3cd48110186d36cf2a35ed36548124c6287abd1a63cf5692d481ed8ec85c7"} Sep 09 21:01:27.791664 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.791617 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" event={"ID":"729a2561-4ff3-4315-b771-6c5faacc6482","Type":"ContainerStarted","Data":"5305338255d9d30650beb5aeeee4ebf8dce23541b38103ce7fe281348974f93d"} Sep 09 21:01:27.799014 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.798810 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmbd7" event={"ID":"96d8365d-dc91-4abb-a692-9ace9f2740d3","Type":"ContainerStarted","Data":"006cccb86d1e20b3813af76875e12fcafdf4aed950a4d1974856657e08d5fa9e"} Sep 09 21:01:27.809297 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.809215 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jq8n4" event={"ID":"71b2bab2-31a6-4b4b-9f09-44ffbfd210c6","Type":"ContainerStarted","Data":"ba7b9c4d454acc29c3acbc5ccd46863c399e9bbab3cf0ab79ca3e6246a526116"} Sep 09 21:01:27.817622 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.817442 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sdc89" event={"ID":"76b95cc0-0a15-4c20-bd29-463861438b51","Type":"ContainerStarted","Data":"b34a85ebe35057e0186d35fcf7971e36a566380de89a65612e1e2a08835ac8e4"} Sep 09 21:01:27.828242 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.828212 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mgt9t" event={"ID":"54a81a73-ac90-4e0b-894e-4b14e0069156","Type":"ContainerStarted","Data":"45a7e5f5c812e366d6daf1651f24af97d1457c7aceda28f3b71201ea83caa5c2"} Sep 09 21:01:27.830591 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.830541 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" event={"ID":"e1caf595-b108-4c86-bde5-d032dbb6e7d4","Type":"ContainerStarted","Data":"73a100a05b37c4f9326a31dcb2df87f5a3d899f9cd7afcad249e634f677a91ee"} Sep 09 21:01:27.841617 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.841587 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-472cd" event={"ID":"e1b87a69-3799-43a2-8e4e-deffdbe8ae09","Type":"ContainerStarted","Data":"bf3fabe207dca0fdae03f00b2fc3116786b4db889b9030a12f57972eb76cfaae"} Sep 09 21:01:27.847719 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:27.847682 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fxqcd" event={"ID":"1c8a253b-db72-4c92-9f00-f1ebf4d22a68","Type":"ContainerStarted","Data":"72220c58ee1c0240dd41d4e8b68c056ea3eef2e9dee3736d0c4525ca77edff66"} Sep 09 21:01:28.163411 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:28.163371 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs\") pod \"network-metrics-daemon-mxztj\" (UID: \"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41\") " pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:28.163607 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:28.163534 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:28.163607 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:28.163596 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs podName:bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:30.163578154 +0000 UTC m=+6.025100528 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs") pod "network-metrics-daemon-mxztj" (UID: "bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:28.264636 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:28.264598 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:28.264830 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:28.264760 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:28.264830 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:28.264823 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret podName:253428b3-7b9b-474c-b3eb-f8b8f63b9671 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:30.264804029 +0000 UTC m=+6.126326404 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret") pod "global-pull-secret-syncer-gj6c4" (UID: "253428b3-7b9b-474c-b3eb-f8b8f63b9671") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:28.365209 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:28.365173 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xscn\" (UniqueName: \"kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn\") pod \"network-check-target-xbvrp\" (UID: \"e717cc32-b9ce-46d2-b624-1391699be568\") " pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:28.365402 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:28.365353 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:01:28.365402 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:28.365377 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:01:28.365402 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:28.365390 2560 projected.go:196] Error preparing data for projected volume kube-api-access-5xscn for pod openshift-network-diagnostics/network-check-target-xbvrp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:28.365579 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:28.365447 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn podName:e717cc32-b9ce-46d2-b624-1391699be568 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:30.365426455 +0000 UTC m=+6.226948849 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5xscn" (UniqueName: "kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn") pod "network-check-target-xbvrp" (UID: "e717cc32-b9ce-46d2-b624-1391699be568") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:28.598976 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:28.598907 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:56:27 +0000 UTC" deadline="2028-04-18 03:30:06.754019214 +0000 UTC" Sep 09 21:01:28.598976 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:28.598973 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14070h28m38.155050434s" Sep 09 21:01:28.777042 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:28.774737 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:28.777042 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:28.774743 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:28.777042 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:28.775015 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:28.777042 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:28.775129 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:29.775370 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:29.774813 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:29.775370 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:29.774958 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:29.877509 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:29.876323 2560 generic.go:356] "Generic (PLEG): container finished" podID="6157f6ca1f732b195484b20e7d38cdee" containerID="7f90ba0c76e59b3b04ee2ae87315425e771b977e6e896e300eb02cd32bbb9ba1" exitCode=0 Sep 09 21:01:29.877509 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:29.876385 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" event={"ID":"6157f6ca1f732b195484b20e7d38cdee","Type":"ContainerDied","Data":"7f90ba0c76e59b3b04ee2ae87315425e771b977e6e896e300eb02cd32bbb9ba1"} Sep 09 21:01:30.181738 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:30.180139 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs\") pod \"network-metrics-daemon-mxztj\" (UID: \"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41\") " pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:30.181738 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:30.180328 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:30.181738 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:30.180393 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs podName:bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:34.180372454 +0000 UTC m=+10.041894850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs") pod "network-metrics-daemon-mxztj" (UID: "bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:30.281029 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:30.280987 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:30.281231 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:30.281181 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:30.281313 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:30.281267 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret podName:253428b3-7b9b-474c-b3eb-f8b8f63b9671 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:34.281245328 +0000 UTC m=+10.142767724 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret") pod "global-pull-secret-syncer-gj6c4" (UID: "253428b3-7b9b-474c-b3eb-f8b8f63b9671") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:30.382223 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:30.382179 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xscn\" (UniqueName: \"kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn\") pod \"network-check-target-xbvrp\" (UID: \"e717cc32-b9ce-46d2-b624-1391699be568\") " pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:30.382475 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:30.382409 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:01:30.382475 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:30.382431 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:01:30.382475 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:30.382444 2560 projected.go:196] Error preparing data for projected volume kube-api-access-5xscn for pod openshift-network-diagnostics/network-check-target-xbvrp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:30.382644 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:30.382508 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn podName:e717cc32-b9ce-46d2-b624-1391699be568 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:34.382488278 +0000 UTC m=+10.244010656 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5xscn" (UniqueName: "kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn") pod "network-check-target-xbvrp" (UID: "e717cc32-b9ce-46d2-b624-1391699be568") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:30.774536 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:30.774496 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:30.774715 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:30.774661 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:30.774937 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:30.774912 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:30.775082 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:30.775055 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:31.774689 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:31.774576 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:31.775221 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:31.774737 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:32.774522 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:32.774486 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:32.774729 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:32.774628 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:32.775262 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:32.775237 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:32.775394 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:32.775367 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:33.774557 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:33.774325 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:33.774557 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:33.774462 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:34.218365 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:34.218319 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs\") pod \"network-metrics-daemon-mxztj\" (UID: \"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41\") " pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:34.218571 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:34.218479 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:34.218571 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:34.218548 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs podName:bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:42.218528396 +0000 UTC m=+18.080050782 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs") pod "network-metrics-daemon-mxztj" (UID: "bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:34.320124 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:34.319667 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:34.320124 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:34.319832 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:34.320124 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:34.319910 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret podName:253428b3-7b9b-474c-b3eb-f8b8f63b9671 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:42.319889779 +0000 UTC m=+18.181412160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret") pod "global-pull-secret-syncer-gj6c4" (UID: "253428b3-7b9b-474c-b3eb-f8b8f63b9671") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:34.420223 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:34.420186 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xscn\" (UniqueName: \"kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn\") pod \"network-check-target-xbvrp\" (UID: \"e717cc32-b9ce-46d2-b624-1391699be568\") " pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:34.420422 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:34.420389 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:01:34.420422 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:34.420409 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:01:34.420422 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:34.420422 2560 projected.go:196] Error preparing data for projected volume kube-api-access-5xscn for pod openshift-network-diagnostics/network-check-target-xbvrp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:34.420592 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:34.420490 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn podName:e717cc32-b9ce-46d2-b624-1391699be568 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:42.420471407 +0000 UTC m=+18.281993795 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5xscn" (UniqueName: "kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn") pod "network-check-target-xbvrp" (UID: "e717cc32-b9ce-46d2-b624-1391699be568") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:34.775737 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:34.775206 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:34.775737 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:34.775345 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:34.776297 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:34.775767 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:34.776297 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:34.775938 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:35.774216 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:35.774142 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:35.774403 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:35.774291 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:36.775241 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:36.774564 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:36.775241 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:36.774564 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:36.775241 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:36.774723 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:36.775241 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:36.774845 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:37.773624 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:37.773590 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:37.773805 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:37.773723 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:38.773647 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:38.773617 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:38.774085 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:38.773722 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:38.774085 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:38.773808 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:38.774085 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:38.773927 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:39.774058 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:39.774014 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:39.774457 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:39.774185 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:40.774190 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:40.774149 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:40.774746 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:40.774197 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:40.774746 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:40.774304 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:40.774746 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:40.774409 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:41.774467 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:41.774430 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:41.774891 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:41.774552 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:42.287014 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:42.286972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs\") pod \"network-metrics-daemon-mxztj\" (UID: \"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41\") " pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:42.287201 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:42.287132 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:42.287267 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:42.287211 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs podName:bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:58.287190135 +0000 UTC m=+34.148712533 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs") pod "network-metrics-daemon-mxztj" (UID: "bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:42.388234 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:42.388187 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:42.388377 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:42.388313 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:42.388377 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:42.388367 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret podName:253428b3-7b9b-474c-b3eb-f8b8f63b9671 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:58.388353059 +0000 UTC m=+34.249875432 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret") pod "global-pull-secret-syncer-gj6c4" (UID: "253428b3-7b9b-474c-b3eb-f8b8f63b9671") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:42.489347 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:42.489310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xscn\" (UniqueName: \"kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn\") pod \"network-check-target-xbvrp\" (UID: \"e717cc32-b9ce-46d2-b624-1391699be568\") " pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:42.489490 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:42.489474 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:01:42.489529 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:42.489496 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:01:42.489529 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:42.489506 2560 projected.go:196] Error preparing data for projected volume kube-api-access-5xscn for pod openshift-network-diagnostics/network-check-target-xbvrp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:42.489587 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:42.489555 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn podName:e717cc32-b9ce-46d2-b624-1391699be568 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:58.48954199 +0000 UTC m=+34.351064363 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5xscn" (UniqueName: "kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn") pod "network-check-target-xbvrp" (UID: "e717cc32-b9ce-46d2-b624-1391699be568") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:42.774695 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:42.774579 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:42.775191 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:42.774718 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:42.775191 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:42.775136 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:42.775291 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:42.775265 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:43.774502 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:43.774460 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:43.774684 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:43.774608 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:44.774466 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:44.774436 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:44.774466 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:44.774460 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:44.774952 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:44.774527 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:44.774952 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:44.774663 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:45.773749 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:45.773714 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:45.773913 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:45.773837 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:46.774603 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.774298 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:46.775023 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.774304 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:46.775023 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:46.774695 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:46.775023 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:46.774786 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:46.910863 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.910833 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal" event={"ID":"1ea201ea53f48274b482ba7553442dfe","Type":"ContainerStarted","Data":"782086f0a56f8cf538e94298c47edab5ba40e7655a67be0538180c7f3dc0bae2"} Sep 09 21:01:46.912346 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.912317 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jq8n4" event={"ID":"71b2bab2-31a6-4b4b-9f09-44ffbfd210c6","Type":"ContainerStarted","Data":"78d4828a329fcc09f0a712e2773733acd61d8fb8aa2ff5c5bea4fbeb5b715e7c"} Sep 09 21:01:46.913734 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.913705 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sdc89" event={"ID":"76b95cc0-0a15-4c20-bd29-463861438b51","Type":"ContainerStarted","Data":"28e3fa42021858a1e8530864cb9f719022a58e9419b4f90c080edb7ecbee6ebf"} Sep 09 21:01:46.915312 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.915292 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mgt9t" event={"ID":"54a81a73-ac90-4e0b-894e-4b14e0069156","Type":"ContainerStarted","Data":"b445d1e46ad72d7fb848b72285df311cfa3f5cd28d68e6d4cb256da7815a48ae"} Sep 09 21:01:46.916905 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.916881 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" event={"ID":"e1caf595-b108-4c86-bde5-d032dbb6e7d4","Type":"ContainerStarted","Data":"91672316133bedddd9705bc51329d985d7516dc921901add0a35ec4377edfe09"} Sep 09 21:01:46.918295 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.918276 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-472cd" event={"ID":"e1b87a69-3799-43a2-8e4e-deffdbe8ae09","Type":"ContainerStarted","Data":"8be987d8806d64e72926b12bb83c1bceeacdc74dd6d21bc0a37a67ba8bad6b84"} Sep 09 21:01:46.920030 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.920007 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" event={"ID":"6157f6ca1f732b195484b20e7d38cdee","Type":"ContainerStarted","Data":"22d59cbf668733edaf72c2ecc8ecceda72e52e55617816b22a121ebf7eb55531"} Sep 09 21:01:46.922204 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.922185 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" event={"ID":"2a3ad848-e2c1-4909-98d6-f432895c189b","Type":"ContainerStarted","Data":"a9cc5dff5e22056a3fbe36ec9a7ed5b3f57cffb1196832e1ce2ad0f98520caa9"} Sep 09 21:01:46.922296 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.922246 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" event={"ID":"2a3ad848-e2c1-4909-98d6-f432895c189b","Type":"ContainerStarted","Data":"42fb8bfd85bdb3bc5d301319946c61beee97bebb97ae285d1fb390b72f8c9016"} Sep 09 21:01:46.922488 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.922293 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" event={"ID":"2a3ad848-e2c1-4909-98d6-f432895c189b","Type":"ContainerStarted","Data":"3b87b18fba8120198e3a2e457fce10fdf601fbfc5391f2de82bd4731a49735b9"} Sep 09 21:01:46.922488 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.922343 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" event={"ID":"2a3ad848-e2c1-4909-98d6-f432895c189b","Type":"ContainerStarted","Data":"aacdae772f65bc5d57167c9561c77257d795d511c0642c2d7256d358829cf070"} Sep 09 21:01:46.923753 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.923729 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" event={"ID":"729a2561-4ff3-4315-b771-6c5faacc6482","Type":"ContainerStarted","Data":"0710fd6deb94aff909c21bb5d34a3b8a05c1cd63564d4763a35a96f8efee5309"} Sep 09 21:01:46.925311 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.925285 2560 generic.go:356] "Generic (PLEG): container finished" podID="96d8365d-dc91-4abb-a692-9ace9f2740d3" containerID="423d18c4dd5420dd6a2df8c041e4b992695f14d151843ccc7e7378148f66ef23" exitCode=0 Sep 09 21:01:46.925384 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.925323 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmbd7" event={"ID":"96d8365d-dc91-4abb-a692-9ace9f2740d3","Type":"ContainerDied","Data":"423d18c4dd5420dd6a2df8c041e4b992695f14d151843ccc7e7378148f66ef23"} Sep 09 21:01:46.945138 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.945016 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal" podStartSLOduration=20.944999022 podStartE2EDuration="20.944999022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:46.927661915 +0000 UTC m=+22.789184311" watchObservedRunningTime="2026-09-09 21:01:46.944999022 +0000 UTC m=+22.806521417" Sep 09 21:01:46.967482 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.967435 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-472cd" podStartSLOduration=6.388114469 podStartE2EDuration="21.96741821s" totalImagesPullingTime="15.579303741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:46.945659625 +0000 UTC m=+22.807182020" watchObservedRunningTime="2026-09-09 21:01:46.96741821 +0000 UTC m=+22.828940605" Sep 09 21:01:46.967598 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:46.967582 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fm9z8" podStartSLOduration=2.500396298 podStartE2EDuration="21.967578127s" totalImagesPullingTime="19.467181829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:46.967306773 +0000 UTC m=+22.828829169" watchObservedRunningTime="2026-09-09 21:01:46.967578127 +0000 UTC m=+22.829100521" Sep 09 21:01:47.001192 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:47.001144 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sdc89" podStartSLOduration=2.574531219 podStartE2EDuration="22.001128332s" totalImagesPullingTime="19.426597113s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:47.000700268 +0000 UTC m=+22.862222684" watchObservedRunningTime="2026-09-09 21:01:47.001128332 +0000 UTC m=+22.862650727" Sep 09 21:01:47.001290 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:47.001242 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mgt9t" podStartSLOduration=2.511655388 podStartE2EDuration="22.001237474s" totalImagesPullingTime="19.489582086s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:46.985448669 +0000 UTC m=+22.846971065" watchObservedRunningTime="2026-09-09 21:01:47.001237474 +0000 UTC m=+22.862759869" Sep 09 21:01:47.075666 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:47.075616 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jq8n4" podStartSLOduration=1.6581986949999998 podStartE2EDuration="21.075599484s" totalImagesPullingTime="19.417400789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:47.038755448 +0000 UTC m=+22.900277842" watchObservedRunningTime="2026-09-09 21:01:47.075599484 +0000 UTC m=+22.937121882" Sep 09 21:01:47.107801 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:47.107705 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" podStartSLOduration=21.107682953 podStartE2EDuration="21.107682953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:47.075796956 +0000 UTC m=+22.937319351" watchObservedRunningTime="2026-09-09 21:01:47.107682953 +0000 UTC m=+22.969205349" Sep 09 21:01:47.773731 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:47.773698 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:47.773951 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:47.773836 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:47.929546 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:47.929459 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fxqcd" event={"ID":"1c8a253b-db72-4c92-9f00-f1ebf4d22a68","Type":"ContainerStarted","Data":"f7e43c8c37f44dd65b583fe65f984d66b7a67fd906c0248f6e1861a99d3dc661"} Sep 09 21:01:47.935527 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:47.934786 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" event={"ID":"2a3ad848-e2c1-4909-98d6-f432895c189b","Type":"ContainerStarted","Data":"7d705646487ac0f8eee0f524f0d0b85b6f8565a6b8d0dd0f17de16e056ba83e1"} Sep 09 21:01:47.935527 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:47.934859 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" event={"ID":"2a3ad848-e2c1-4909-98d6-f432895c189b","Type":"ContainerStarted","Data":"a90737fa3954a0e0ddeaaa2a420e438da842c628872a2475139291fb59c8e47c"} Sep 09 21:01:48.107236 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:48.107034 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 09 21:01:48.549312 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:48.549275 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-472cd" Sep 09 21:01:48.549847 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:48.549828 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-472cd" Sep 09 21:01:48.567412 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:48.567354 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fxqcd" podStartSLOduration=4.104175399 podStartE2EDuration="23.567332945s" totalImagesPullingTime="19.463157546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:47.95917414 +0000 UTC m=+23.820696535" watchObservedRunningTime="2026-09-09 21:01:48.567332945 +0000 UTC m=+24.428855340" Sep 09 21:01:48.745483 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:48.745342 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T21:01:48.107229092Z","UUID":"2892110d-bd07-4bb4-abdb-8dca581a585f","Handler":null,"Name":"","Endpoint":""} Sep 09 21:01:48.746963 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:48.746939 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 09 21:01:48.747112 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:48.746974 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 09 21:01:48.774014 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:48.773984 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:48.774187 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:48.773987 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:48.774187 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:48.774108 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:48.774187 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:48.774173 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:48.938967 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:48.938837 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" event={"ID":"729a2561-4ff3-4315-b771-6c5faacc6482","Type":"ContainerStarted","Data":"a4f9cb8a50a7f52864383096f9b022af62b2f8d1fe333a02ec210376a6c269f1"} Sep 09 21:01:49.774391 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:49.774364 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:49.774541 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:49.774490 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:49.945378 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:49.945340 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" event={"ID":"2a3ad848-e2c1-4909-98d6-f432895c189b","Type":"ContainerStarted","Data":"468802a930e4f05e3d93b344fdee1bf6e5f799031be06d8f46b59e6702cb63c4"} Sep 09 21:01:49.947541 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:49.947510 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" event={"ID":"729a2561-4ff3-4315-b771-6c5faacc6482","Type":"ContainerStarted","Data":"7007816dc290e59c5da10dd1111e73f786bb579fbce5d50309384a4580e5ff34"} Sep 09 21:01:49.947541 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:49.947536 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 09 21:01:49.971257 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:49.971102 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4r7r" podStartSLOduration=2.538491088 podStartE2EDuration="24.971080717s" totalImagesPullingTime="22.432589629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:49.970136689 +0000 UTC m=+25.831659084" watchObservedRunningTime="2026-09-09 21:01:49.971080717 +0000 UTC m=+25.832603114" Sep 09 21:01:50.774214 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:50.774176 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:50.774404 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:50.774175 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:50.774404 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:50.774303 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:50.774519 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:50.774406 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:51.774155 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:51.774125 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:51.774466 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:51.774240 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:51.958004 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:51.957730 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" event={"ID":"2a3ad848-e2c1-4909-98d6-f432895c189b","Type":"ContainerStarted","Data":"ed2dd78e8e55a3b4308ab5875ffbd07cd160ae5171325d3f3537bbceb5252c49"} Sep 09 21:01:51.958255 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:51.958139 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:51.958255 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:51.958213 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:51.958364 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:51.958261 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:51.960228 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:51.960092 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmbd7" event={"ID":"96d8365d-dc91-4abb-a692-9ace9f2740d3","Type":"ContainerStarted","Data":"b299964c5b5159877abdab4a7c1da5a897bf5a3f30fdcdc3dc018c5eb3d87f97"} Sep 09 21:01:51.974199 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:51.974171 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:51.974386 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:51.974374 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:01:51.995981 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:51.995903 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-472cd" Sep 09 21:01:51.996126 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:51.996080 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 09 21:01:51.996517 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:51.996494 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-472cd" Sep 09 21:01:51.997005 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:51.996966 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" podStartSLOduration=7.518247732 podStartE2EDuration="26.996952882s" totalImagesPullingTime="19.47870515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:51.996339504 +0000 UTC m=+27.857861901" watchObservedRunningTime="2026-09-09 21:01:51.996952882 +0000 UTC m=+27.858475277" Sep 09 21:01:52.774397 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:52.774361 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:52.774894 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:52.774472 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:52.774894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:52.774524 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:52.774894 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:52.774612 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:52.966401 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:52.966364 2560 generic.go:356] "Generic (PLEG): container finished" podID="96d8365d-dc91-4abb-a692-9ace9f2740d3" containerID="b299964c5b5159877abdab4a7c1da5a897bf5a3f30fdcdc3dc018c5eb3d87f97" exitCode=0 Sep 09 21:01:52.966561 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:52.966419 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmbd7" event={"ID":"96d8365d-dc91-4abb-a692-9ace9f2740d3","Type":"ContainerDied","Data":"b299964c5b5159877abdab4a7c1da5a897bf5a3f30fdcdc3dc018c5eb3d87f97"} Sep 09 21:01:53.678491 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:53.678440 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mxztj"] Sep 09 21:01:53.678691 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:53.678575 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:53.678767 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:53.678697 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:53.680797 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:53.680766 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gj6c4"] Sep 09 21:01:53.680939 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:53.680878 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:53.680995 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:53.680980 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:53.683527 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:53.683459 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xbvrp"] Sep 09 21:01:53.683664 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:53.683537 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:53.683664 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:53.683622 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:54.776634 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:54.776110 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:54.776634 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:54.776272 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:54.972856 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:54.972819 2560 generic.go:356] "Generic (PLEG): container finished" podID="96d8365d-dc91-4abb-a692-9ace9f2740d3" containerID="dfe27cf7302b001c8414aaa845a4786ce2787d2f8beb5c3307fbdff28cbc3f89" exitCode=0 Sep 09 21:01:54.972856 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:54.972860 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmbd7" event={"ID":"96d8365d-dc91-4abb-a692-9ace9f2740d3","Type":"ContainerDied","Data":"dfe27cf7302b001c8414aaa845a4786ce2787d2f8beb5c3307fbdff28cbc3f89"} Sep 09 21:01:55.774289 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:55.774257 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:55.774443 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:55.774257 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:55.774443 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:55.774402 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:55.774545 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:55.774452 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:56.774082 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:56.773888 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:56.774437 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:56.774154 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:56.981131 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:56.981094 2560 generic.go:356] "Generic (PLEG): container finished" podID="96d8365d-dc91-4abb-a692-9ace9f2740d3" containerID="62273fd08305d55a244ee513a940b083cf1557e41e459bdb98e391da41218692" exitCode=0 Sep 09 21:01:56.981298 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:56.981143 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmbd7" event={"ID":"96d8365d-dc91-4abb-a692-9ace9f2740d3","Type":"ContainerDied","Data":"62273fd08305d55a244ee513a940b083cf1557e41e459bdb98e391da41218692"} Sep 09 21:01:57.773983 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:57.773944 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:57.774162 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:57.773950 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:57.774162 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:57.774081 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj6c4" podUID="253428b3-7b9b-474c-b3eb-f8b8f63b9671" Sep 09 21:01:57.774162 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:57.774130 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxztj" podUID="bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41" Sep 09 21:01:58.302656 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:58.302600 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs\") pod \"network-metrics-daemon-mxztj\" (UID: \"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41\") " pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:58.302942 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:58.302788 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:58.302942 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:58.302868 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs podName:bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:30.302848282 +0000 UTC m=+66.164370655 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs") pod "network-metrics-daemon-mxztj" (UID: "bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:58.403336 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:58.403298 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:58.403509 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:58.403486 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:58.403578 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:58.403566 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret podName:253428b3-7b9b-474c-b3eb-f8b8f63b9671 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:30.403544715 +0000 UTC m=+66.265067106 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret") pod "global-pull-secret-syncer-gj6c4" (UID: "253428b3-7b9b-474c-b3eb-f8b8f63b9671") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:58.504298 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:58.504266 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xscn\" (UniqueName: \"kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn\") pod \"network-check-target-xbvrp\" (UID: \"e717cc32-b9ce-46d2-b624-1391699be568\") " pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:58.504471 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:58.504410 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:01:58.504471 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:58.504425 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:01:58.504471 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:58.504437 2560 projected.go:196] Error preparing data for projected volume kube-api-access-5xscn for pod openshift-network-diagnostics/network-check-target-xbvrp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:58.504614 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:58.504490 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn podName:e717cc32-b9ce-46d2-b624-1391699be568 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:30.50447683 +0000 UTC m=+66.365999203 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5xscn" (UniqueName: "kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn") pod "network-check-target-xbvrp" (UID: "e717cc32-b9ce-46d2-b624-1391699be568") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:58.774229 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:58.774198 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:01:58.774680 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:58.774321 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbvrp" podUID="e717cc32-b9ce-46d2-b624-1391699be568" Sep 09 21:01:59.423012 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.422982 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeReady" Sep 09 21:01:59.423183 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.423152 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 09 21:01:59.475614 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.475576 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-777dbd79f8-87p9r"] Sep 09 21:01:59.477616 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.477598 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.481361 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.481336 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 09 21:01:59.481361 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.481349 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 09 21:01:59.481503 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.481416 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 09 21:01:59.481814 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.481736 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xcsfw\"" Sep 09 21:01:59.491059 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.488718 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 09 21:01:59.498111 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.498084 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5j4zn"] Sep 09 21:01:59.500201 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.500145 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-777dbd79f8-87p9r"] Sep 09 21:01:59.500201 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.500180 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nv9wb"] Sep 09 21:01:59.500355 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.500247 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5j4zn" Sep 09 21:01:59.502027 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.502005 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nv9wb" Sep 09 21:01:59.511602 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.511581 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 09 21:01:59.511710 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.511617 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 09 21:01:59.511785 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.511726 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 09 21:01:59.511865 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.511848 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9vjrs\"" Sep 09 21:01:59.511926 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.511857 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 09 21:01:59.511926 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.511922 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 09 21:01:59.512149 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.512134 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c4rmk\"" Sep 09 21:01:59.518127 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.518107 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nv9wb"] Sep 09 21:01:59.523117 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.523094 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5j4zn"] Sep 09 21:01:59.611359 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.611271 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fabe4da8-a1c9-4e4b-aff7-65756b9de477-ca-trust-extracted\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.611359 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.611312 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:01:59.611550 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.611404 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fabe4da8-a1c9-4e4b-aff7-65756b9de477-image-registry-private-configuration\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.611550 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.611439 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fabe4da8-a1c9-4e4b-aff7-65756b9de477-installation-pull-secrets\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.611550 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.611460 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-bound-sa-token\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.611550 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.611483 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k9jr\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-kube-api-access-6k9jr\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.611550 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.611510 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6qbj\" (UniqueName: \"kubernetes.io/projected/f73f9bb9-8de7-4211-a99d-758f7c2721b5-kube-api-access-q6qbj\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:01:59.611776 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.611564 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-certificates\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.611776 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.611585 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fabe4da8-a1c9-4e4b-aff7-65756b9de477-trusted-ca\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.611776 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.611609 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f73f9bb9-8de7-4211-a99d-758f7c2721b5-config-volume\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:01:59.611776 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.611637 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.611776 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.611682 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f73f9bb9-8de7-4211-a99d-758f7c2721b5-tmp-dir\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:01:59.611776 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.611697 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/588a22bb-35b2-43cc-86fa-50afff61eb82-cert\") pod \"ingress-canary-nv9wb\" (UID: \"588a22bb-35b2-43cc-86fa-50afff61eb82\") " pod="openshift-ingress-canary/ingress-canary-nv9wb" Sep 09 21:01:59.611776 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.611724 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhwkn\" (UniqueName: \"kubernetes.io/projected/588a22bb-35b2-43cc-86fa-50afff61eb82-kube-api-access-nhwkn\") pod \"ingress-canary-nv9wb\" (UID: \"588a22bb-35b2-43cc-86fa-50afff61eb82\") " pod="openshift-ingress-canary/ingress-canary-nv9wb" Sep 09 21:01:59.712336 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.712302 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-certificates\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.712499 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.712338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fabe4da8-a1c9-4e4b-aff7-65756b9de477-trusted-ca\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.712499 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.712364 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f73f9bb9-8de7-4211-a99d-758f7c2721b5-config-volume\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:01:59.712499 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.712393 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.712499 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.712427 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f73f9bb9-8de7-4211-a99d-758f7c2721b5-tmp-dir\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:01:59.712499 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.712449 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/588a22bb-35b2-43cc-86fa-50afff61eb82-cert\") pod \"ingress-canary-nv9wb\" (UID: \"588a22bb-35b2-43cc-86fa-50afff61eb82\") " pod="openshift-ingress-canary/ingress-canary-nv9wb" Sep 09 21:01:59.712499 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.712482 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhwkn\" (UniqueName: \"kubernetes.io/projected/588a22bb-35b2-43cc-86fa-50afff61eb82-kube-api-access-nhwkn\") pod \"ingress-canary-nv9wb\" (UID: \"588a22bb-35b2-43cc-86fa-50afff61eb82\") " pod="openshift-ingress-canary/ingress-canary-nv9wb" Sep 09 21:01:59.712789 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.712515 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fabe4da8-a1c9-4e4b-aff7-65756b9de477-ca-trust-extracted\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.712789 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.712561 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:01:59.712789 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.712617 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fabe4da8-a1c9-4e4b-aff7-65756b9de477-image-registry-private-configuration\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.712789 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.712646 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fabe4da8-a1c9-4e4b-aff7-65756b9de477-installation-pull-secrets\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.712789 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:59.712645 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:01:59.712789 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:59.712675 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-777dbd79f8-87p9r: secret "image-registry-tls" not found Sep 09 21:01:59.712789 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:59.712732 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls podName:fabe4da8-a1c9-4e4b-aff7-65756b9de477 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:00.212714257 +0000 UTC m=+36.074236650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls") pod "image-registry-777dbd79f8-87p9r" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477") : secret "image-registry-tls" not found Sep 09 21:01:59.713153 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.713036 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f73f9bb9-8de7-4211-a99d-758f7c2721b5-config-volume\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:01:59.713153 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.713071 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-certificates\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.713153 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.713101 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fabe4da8-a1c9-4e4b-aff7-65756b9de477-ca-trust-extracted\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.713290 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:59.713166 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:01:59.713290 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:01:59.713209 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls podName:f73f9bb9-8de7-4211-a99d-758f7c2721b5 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:00.213193319 +0000 UTC m=+36.074715692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls") pod "dns-default-5j4zn" (UID: "f73f9bb9-8de7-4211-a99d-758f7c2721b5") : secret "dns-default-metrics-tls" not found Sep 09 21:01:59.713398 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.713307 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-bound-sa-token\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.713398 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.713349 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k9jr\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-kube-api-access-6k9jr\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.713398 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.713376 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6qbj\" (UniqueName: \"kubernetes.io/projected/f73f9bb9-8de7-4211-a99d-758f7c2721b5-kube-api-access-q6qbj\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:01:59.713538 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.713440 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fabe4da8-a1c9-4e4b-aff7-65756b9de477-trusted-ca\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.713598 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.713563 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f73f9bb9-8de7-4211-a99d-758f7c2721b5-tmp-dir\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:01:59.717551 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.717526 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fabe4da8-a1c9-4e4b-aff7-65756b9de477-installation-pull-secrets\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.717551 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.717542 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/588a22bb-35b2-43cc-86fa-50afff61eb82-cert\") pod \"ingress-canary-nv9wb\" (UID: \"588a22bb-35b2-43cc-86fa-50afff61eb82\") " pod="openshift-ingress-canary/ingress-canary-nv9wb" Sep 09 21:01:59.717705 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.717548 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fabe4da8-a1c9-4e4b-aff7-65756b9de477-image-registry-private-configuration\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.731777 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.731749 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6qbj\" (UniqueName: \"kubernetes.io/projected/f73f9bb9-8de7-4211-a99d-758f7c2721b5-kube-api-access-q6qbj\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:01:59.733147 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.733121 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhwkn\" (UniqueName: \"kubernetes.io/projected/588a22bb-35b2-43cc-86fa-50afff61eb82-kube-api-access-nhwkn\") pod \"ingress-canary-nv9wb\" (UID: \"588a22bb-35b2-43cc-86fa-50afff61eb82\") " pod="openshift-ingress-canary/ingress-canary-nv9wb" Sep 09 21:01:59.735449 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.735407 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k9jr\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-kube-api-access-6k9jr\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.746817 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.746793 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-bound-sa-token\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:01:59.774429 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.774406 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:01:59.774771 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.774429 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:01:59.778233 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.778214 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 21:01:59.778337 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.778234 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 21:01:59.781225 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.781204 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rgvxk\"" Sep 09 21:01:59.817277 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:01:59.817254 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nv9wb" Sep 09 21:02:00.102062 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:00.101876 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nv9wb"] Sep 09 21:02:00.217922 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:00.217877 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:02:00.218136 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:00.217994 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:02:00.218136 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:00.218077 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:02:00.218256 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:00.218150 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:02:00.218256 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:00.218167 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-777dbd79f8-87p9r: secret "image-registry-tls" not found Sep 09 21:02:00.218256 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:00.218155 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls podName:f73f9bb9-8de7-4211-a99d-758f7c2721b5 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:01.218131035 +0000 UTC m=+37.079653411 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls") pod "dns-default-5j4zn" (UID: "f73f9bb9-8de7-4211-a99d-758f7c2721b5") : secret "dns-default-metrics-tls" not found Sep 09 21:02:00.218256 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:00.218237 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls podName:fabe4da8-a1c9-4e4b-aff7-65756b9de477 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:01.218220403 +0000 UTC m=+37.079742785 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls") pod "image-registry-777dbd79f8-87p9r" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477") : secret "image-registry-tls" not found Sep 09 21:02:00.773879 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:00.773840 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:02:00.776755 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:00.776734 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zv66x\"" Sep 09 21:02:00.776755 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:00.776745 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 21:02:00.777869 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:00.777844 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 21:02:01.227855 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:01.227798 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:02:01.228086 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:01.227882 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:02:01.228086 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:01.228058 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:02:01.228202 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:01.228120 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls podName:f73f9bb9-8de7-4211-a99d-758f7c2721b5 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:03.228102384 +0000 UTC m=+39.089624759 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls") pod "dns-default-5j4zn" (UID: "f73f9bb9-8de7-4211-a99d-758f7c2721b5") : secret "dns-default-metrics-tls" not found Sep 09 21:02:01.228425 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:01.228398 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:02:01.228425 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:01.228424 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-777dbd79f8-87p9r: secret "image-registry-tls" not found Sep 09 21:02:01.228604 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:01.228483 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls podName:fabe4da8-a1c9-4e4b-aff7-65756b9de477 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:03.228465117 +0000 UTC m=+39.089987494 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls") pod "image-registry-777dbd79f8-87p9r" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477") : secret "image-registry-tls" not found Sep 09 21:02:01.559908 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:02:01.559833 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod588a22bb_35b2_43cc_86fa_50afff61eb82.slice/crio-ff60cd8d95b2dd026969a0d89c16a284bae3df726dfaad42b39f72f7ae553f6b WatchSource:0}: Error finding container ff60cd8d95b2dd026969a0d89c16a284bae3df726dfaad42b39f72f7ae553f6b: Status 404 returned error can't find the container with id ff60cd8d95b2dd026969a0d89c16a284bae3df726dfaad42b39f72f7ae553f6b Sep 09 21:02:01.995814 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:01.995780 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nv9wb" event={"ID":"588a22bb-35b2-43cc-86fa-50afff61eb82","Type":"ContainerStarted","Data":"ff60cd8d95b2dd026969a0d89c16a284bae3df726dfaad42b39f72f7ae553f6b"} Sep 09 21:02:01.998199 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:01.998175 2560 generic.go:356] "Generic (PLEG): container finished" podID="96d8365d-dc91-4abb-a692-9ace9f2740d3" containerID="d9c0925e3a8d2ed209238440a3c5cbb1271f7b95bb65ab28b6da4d9502ed8b7a" exitCode=0 Sep 09 21:02:01.998319 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:01.998208 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmbd7" event={"ID":"96d8365d-dc91-4abb-a692-9ace9f2740d3","Type":"ContainerDied","Data":"d9c0925e3a8d2ed209238440a3c5cbb1271f7b95bb65ab28b6da4d9502ed8b7a"} Sep 09 21:02:02.235101 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:02.235067 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:02.369588 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:02.369496 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:02.470451 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:02.470418 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:02.883276 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:02.883241 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:03.003163 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:03.003126 2560 generic.go:356] "Generic (PLEG): container finished" podID="96d8365d-dc91-4abb-a692-9ace9f2740d3" containerID="35dd081ae4003a66825b508946a3c586a3121f99b4447dc8c178216c78eeac24" exitCode=0 Sep 09 21:02:03.003570 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:03.003183 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmbd7" event={"ID":"96d8365d-dc91-4abb-a692-9ace9f2740d3","Type":"ContainerDied","Data":"35dd081ae4003a66825b508946a3c586a3121f99b4447dc8c178216c78eeac24"} Sep 09 21:02:03.243552 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:03.243522 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:02:03.243697 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:03.243570 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:02:03.243697 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:03.243663 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:02:03.243697 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:03.243670 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:02:03.243697 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:03.243688 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-777dbd79f8-87p9r: secret "image-registry-tls" not found Sep 09 21:02:03.243822 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:03.243713 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls podName:f73f9bb9-8de7-4211-a99d-758f7c2721b5 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:07.24370001 +0000 UTC m=+43.105222383 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls") pod "dns-default-5j4zn" (UID: "f73f9bb9-8de7-4211-a99d-758f7c2721b5") : secret "dns-default-metrics-tls" not found Sep 09 21:02:03.243822 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:03.243732 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls podName:fabe4da8-a1c9-4e4b-aff7-65756b9de477 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:07.243719878 +0000 UTC m=+43.105242255 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls") pod "image-registry-777dbd79f8-87p9r" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477") : secret "image-registry-tls" not found Sep 09 21:02:03.303927 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:03.303877 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:04.010775 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:04.010736 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hmbd7" event={"ID":"96d8365d-dc91-4abb-a692-9ace9f2740d3","Type":"ContainerStarted","Data":"5d00929bc295c861e70337cca6be87ad6eea28f3de148911d01921345a6cebd6"} Sep 09 21:02:04.191871 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:04.191832 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:04.261204 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:04.261097 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:04.332265 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:04.332222 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:04.810138 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:04.810090 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:06.017778 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:06.017732 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nv9wb" event={"ID":"588a22bb-35b2-43cc-86fa-50afff61eb82","Type":"ContainerStarted","Data":"305e2b67d3e6a0de72f03045984d884e2bb11549b45a727d57376d066abda78d"} Sep 09 21:02:06.044569 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:06.044509 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hmbd7" podStartSLOduration=9.64028056 podStartE2EDuration="41.044491342s" totalImagesPullingTime="31.404210782s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:04.048977567 +0000 UTC m=+39.910499963" watchObservedRunningTime="2026-09-09 21:02:06.044491342 +0000 UTC m=+41.906013737" Sep 09 21:02:06.044913 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:06.044893 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nv9wb" podStartSLOduration=2.765056137 podStartE2EDuration="7.044885436s" totalImagesPullingTime="4.279829299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:06.04423205 +0000 UTC m=+41.905754445" watchObservedRunningTime="2026-09-09 21:02:06.044885436 +0000 UTC m=+41.906407830" Sep 09 21:02:07.276322 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:07.276283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:02:07.276684 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:07.276341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:02:07.276684 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:07.276433 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:02:07.276684 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:07.276442 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:02:07.276684 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:07.276461 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-777dbd79f8-87p9r: secret "image-registry-tls" not found Sep 09 21:02:07.276684 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:07.276487 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls podName:f73f9bb9-8de7-4211-a99d-758f7c2721b5 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:15.276474731 +0000 UTC m=+51.137997105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls") pod "dns-default-5j4zn" (UID: "f73f9bb9-8de7-4211-a99d-758f7c2721b5") : secret "dns-default-metrics-tls" not found Sep 09 21:02:07.276684 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:07.276509 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls podName:fabe4da8-a1c9-4e4b-aff7-65756b9de477 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:15.276495389 +0000 UTC m=+51.138017765 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls") pod "image-registry-777dbd79f8-87p9r" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477") : secret "image-registry-tls" not found Sep 09 21:02:11.120342 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:11.120309 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:11.221199 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:11.221166 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:13.664055 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:13.664007 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:13.735370 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:13.735337 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:15.338209 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:15.338165 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:02:15.338745 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:15.338232 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:02:15.338745 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:15.338353 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:02:15.338745 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:15.338374 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-777dbd79f8-87p9r: secret "image-registry-tls" not found Sep 09 21:02:15.338745 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:15.338425 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls podName:fabe4da8-a1c9-4e4b-aff7-65756b9de477 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:31.33841001 +0000 UTC m=+67.199932384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls") pod "image-registry-777dbd79f8-87p9r" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477") : secret "image-registry-tls" not found Sep 09 21:02:15.338745 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:15.338443 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:02:15.338745 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:15.338529 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls podName:f73f9bb9-8de7-4211-a99d-758f7c2721b5 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:31.338508646 +0000 UTC m=+67.200031034 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls") pod "dns-default-5j4zn" (UID: "f73f9bb9-8de7-4211-a99d-758f7c2721b5") : secret "dns-default-metrics-tls" not found Sep 09 21:02:15.942890 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:15.942839 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:16.044096 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:16.044063 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:17.666057 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:17.666010 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:17.732785 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:17.732721 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:18.027676 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:18.027644 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:18.109879 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:18.109847 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:18.182438 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:18.182407 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:18.446353 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:18.446264 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:18.547010 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:18.546981 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:18.612771 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:18.612735 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:23.985318 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:23.985285 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhw7b" Sep 09 21:02:24.163781 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:24.163745 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:24.227437 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:24.227401 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:30.341971 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:30.341936 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs\") pod \"network-metrics-daemon-mxztj\" (UID: \"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41\") " pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:02:30.344533 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:30.344512 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 21:02:30.352627 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:30.352603 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 21:02:30.352692 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:30.352681 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs podName:bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41 nodeName:}" failed. No retries permitted until 2026-09-09 21:03:34.352660623 +0000 UTC m=+130.214182996 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs") pod "network-metrics-daemon-mxztj" (UID: "bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41") : secret "metrics-daemon-secret" not found Sep 09 21:02:30.443029 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:30.442993 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:02:30.445808 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:30.445788 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 21:02:30.456802 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:30.456779 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/253428b3-7b9b-474c-b3eb-f8b8f63b9671-original-pull-secret\") pod \"global-pull-secret-syncer-gj6c4\" (UID: \"253428b3-7b9b-474c-b3eb-f8b8f63b9671\") " pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:02:30.543739 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:30.543700 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xscn\" (UniqueName: \"kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn\") pod \"network-check-target-xbvrp\" (UID: \"e717cc32-b9ce-46d2-b624-1391699be568\") " pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:02:30.546419 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:30.546402 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 21:02:30.557217 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:30.557186 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 21:02:30.566909 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:30.566877 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xscn\" (UniqueName: \"kubernetes.io/projected/e717cc32-b9ce-46d2-b624-1391699be568-kube-api-access-5xscn\") pod \"network-check-target-xbvrp\" (UID: \"e717cc32-b9ce-46d2-b624-1391699be568\") " pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:02:30.691564 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:30.691453 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj6c4" Sep 09 21:02:30.786563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:30.786535 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zv66x\"" Sep 09 21:02:30.793897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:30.793861 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:02:30.964814 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:30.964786 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gj6c4"] Sep 09 21:02:30.968072 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:02:30.968028 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod253428b3_7b9b_474c_b3eb_f8b8f63b9671.slice/crio-a616b981b3f2d204f2b2f41427a7ddc7ad1efb0dcd0529a3789db7a3c897f37f WatchSource:0}: Error finding container a616b981b3f2d204f2b2f41427a7ddc7ad1efb0dcd0529a3789db7a3c897f37f: Status 404 returned error can't find the container with id a616b981b3f2d204f2b2f41427a7ddc7ad1efb0dcd0529a3789db7a3c897f37f Sep 09 21:02:31.061379 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:31.061342 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xbvrp"] Sep 09 21:02:31.064403 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:02:31.064375 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode717cc32_b9ce_46d2_b624_1391699be568.slice/crio-5a8ef54e1d2d931776650aae21cb2aa79f7df1eb416d9c8a7a0f06f817541e33 WatchSource:0}: Error finding container 5a8ef54e1d2d931776650aae21cb2aa79f7df1eb416d9c8a7a0f06f817541e33: Status 404 returned error can't find the container with id 5a8ef54e1d2d931776650aae21cb2aa79f7df1eb416d9c8a7a0f06f817541e33 Sep 09 21:02:31.073594 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:31.073566 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xbvrp" event={"ID":"e717cc32-b9ce-46d2-b624-1391699be568","Type":"ContainerStarted","Data":"5a8ef54e1d2d931776650aae21cb2aa79f7df1eb416d9c8a7a0f06f817541e33"} Sep 09 21:02:31.074600 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:31.074578 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gj6c4" event={"ID":"253428b3-7b9b-474c-b3eb-f8b8f63b9671","Type":"ContainerStarted","Data":"a616b981b3f2d204f2b2f41427a7ddc7ad1efb0dcd0529a3789db7a3c897f37f"} Sep 09 21:02:31.351356 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:31.351260 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:02:31.351356 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:31.351310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:02:31.351795 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:31.351433 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:02:31.351795 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:31.351438 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:02:31.351795 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:31.351463 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-777dbd79f8-87p9r: secret "image-registry-tls" not found Sep 09 21:02:31.351795 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:31.351487 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls podName:f73f9bb9-8de7-4211-a99d-758f7c2721b5 nodeName:}" failed. No retries permitted until 2026-09-09 21:03:03.35147172 +0000 UTC m=+99.212994095 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls") pod "dns-default-5j4zn" (UID: "f73f9bb9-8de7-4211-a99d-758f7c2721b5") : secret "dns-default-metrics-tls" not found Sep 09 21:02:31.351795 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:31.351532 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls podName:fabe4da8-a1c9-4e4b-aff7-65756b9de477 nodeName:}" failed. No retries permitted until 2026-09-09 21:03:03.351514132 +0000 UTC m=+99.213036525 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls") pod "image-registry-777dbd79f8-87p9r" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477") : secret "image-registry-tls" not found Sep 09 21:02:34.314366 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.314331 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt"] Sep 09 21:02:34.316158 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.316142 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" Sep 09 21:02:34.318870 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.318837 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:02:34.318870 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.318836 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 09 21:02:34.319344 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.319324 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 09 21:02:34.319448 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.319412 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 09 21:02:34.319982 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.319967 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-clkf5\"" Sep 09 21:02:34.322479 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.322461 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-rrrx7"] Sep 09 21:02:34.324259 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.324245 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.327882 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.327865 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 09 21:02:34.327960 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.327869 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 09 21:02:34.328631 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.328613 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 09 21:02:34.328631 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.328624 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 09 21:02:34.329195 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.329177 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-cdjp4\"" Sep 09 21:02:34.332748 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.332725 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt"] Sep 09 21:02:34.335168 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.335150 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 09 21:02:34.344945 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.344921 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-rrrx7"] Sep 09 21:02:34.373274 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.373238 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t579w\" (UniqueName: \"kubernetes.io/projected/0f2af69c-f46c-412c-ade6-00311e4e59b0-kube-api-access-t579w\") pod \"service-ca-operator-859f94c4b-6qrdt\" (UID: \"0f2af69c-f46c-412c-ade6-00311e4e59b0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" Sep 09 21:02:34.373436 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.373278 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0f2af69c-f46c-412c-ade6-00311e4e59b0-config\") pod \"service-ca-operator-859f94c4b-6qrdt\" (UID: \"0f2af69c-f46c-412c-ade6-00311e4e59b0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" Sep 09 21:02:34.373436 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.373323 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w5xk\" (UniqueName: \"kubernetes.io/projected/2da07b2d-4244-45d8-87e5-ada3a82297fc-kube-api-access-6w5xk\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.373535 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.373442 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2da07b2d-4244-45d8-87e5-ada3a82297fc-serving-cert\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.373535 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.373494 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0f2af69c-f46c-412c-ade6-00311e4e59b0-serving-cert\") pod \"service-ca-operator-859f94c4b-6qrdt\" (UID: \"0f2af69c-f46c-412c-ade6-00311e4e59b0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" Sep 09 21:02:34.373622 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.373535 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2da07b2d-4244-45d8-87e5-ada3a82297fc-tmp\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.373622 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.373560 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2da07b2d-4244-45d8-87e5-ada3a82297fc-snapshots\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.373622 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.373595 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2da07b2d-4244-45d8-87e5-ada3a82297fc-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.373767 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.373622 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2da07b2d-4244-45d8-87e5-ada3a82297fc-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.474813 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.474771 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2da07b2d-4244-45d8-87e5-ada3a82297fc-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.474813 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.474826 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t579w\" (UniqueName: \"kubernetes.io/projected/0f2af69c-f46c-412c-ade6-00311e4e59b0-kube-api-access-t579w\") pod \"service-ca-operator-859f94c4b-6qrdt\" (UID: \"0f2af69c-f46c-412c-ade6-00311e4e59b0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" Sep 09 21:02:34.475107 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.474857 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0f2af69c-f46c-412c-ade6-00311e4e59b0-config\") pod \"service-ca-operator-859f94c4b-6qrdt\" (UID: \"0f2af69c-f46c-412c-ade6-00311e4e59b0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" Sep 09 21:02:34.475107 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.475062 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w5xk\" (UniqueName: \"kubernetes.io/projected/2da07b2d-4244-45d8-87e5-ada3a82297fc-kube-api-access-6w5xk\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.475224 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.475122 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2da07b2d-4244-45d8-87e5-ada3a82297fc-serving-cert\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.475224 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.475161 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0f2af69c-f46c-412c-ade6-00311e4e59b0-serving-cert\") pod \"service-ca-operator-859f94c4b-6qrdt\" (UID: \"0f2af69c-f46c-412c-ade6-00311e4e59b0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" Sep 09 21:02:34.475224 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.475186 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2da07b2d-4244-45d8-87e5-ada3a82297fc-tmp\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.475224 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.475207 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2da07b2d-4244-45d8-87e5-ada3a82297fc-snapshots\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.475414 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.475244 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2da07b2d-4244-45d8-87e5-ada3a82297fc-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.475542 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.475520 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0f2af69c-f46c-412c-ade6-00311e4e59b0-config\") pod \"service-ca-operator-859f94c4b-6qrdt\" (UID: \"0f2af69c-f46c-412c-ade6-00311e4e59b0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" Sep 09 21:02:34.475592 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.475558 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2da07b2d-4244-45d8-87e5-ada3a82297fc-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.476673 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.475901 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2da07b2d-4244-45d8-87e5-ada3a82297fc-tmp\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.476673 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.476115 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2da07b2d-4244-45d8-87e5-ada3a82297fc-snapshots\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.476673 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.476631 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2da07b2d-4244-45d8-87e5-ada3a82297fc-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.480108 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.480085 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0f2af69c-f46c-412c-ade6-00311e4e59b0-serving-cert\") pod \"service-ca-operator-859f94c4b-6qrdt\" (UID: \"0f2af69c-f46c-412c-ade6-00311e4e59b0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" Sep 09 21:02:34.480226 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.480144 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2da07b2d-4244-45d8-87e5-ada3a82297fc-serving-cert\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.483340 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.483316 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t579w\" (UniqueName: \"kubernetes.io/projected/0f2af69c-f46c-412c-ade6-00311e4e59b0-kube-api-access-t579w\") pod \"service-ca-operator-859f94c4b-6qrdt\" (UID: \"0f2af69c-f46c-412c-ade6-00311e4e59b0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" Sep 09 21:02:34.483638 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.483614 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w5xk\" (UniqueName: \"kubernetes.io/projected/2da07b2d-4244-45d8-87e5-ada3a82297fc-kube-api-access-6w5xk\") pod \"insights-operator-76bcd6ccf5-rrrx7\" (UID: \"2da07b2d-4244-45d8-87e5-ada3a82297fc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.583103 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.583003 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:34.625289 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.625253 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" Sep 09 21:02:34.632014 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.631983 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" Sep 09 21:02:34.685014 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.684312 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:34.909020 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.908934 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt"] Sep 09 21:02:34.913823 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:34.913794 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-rrrx7"] Sep 09 21:02:35.707221 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:02:35.707173 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2da07b2d_4244_45d8_87e5_ada3a82297fc.slice/crio-c2084fd51b8ee9bc99d3d35edf3b0d89970efa0d39636bac17b14d6e07e53d92 WatchSource:0}: Error finding container c2084fd51b8ee9bc99d3d35edf3b0d89970efa0d39636bac17b14d6e07e53d92: Status 404 returned error can't find the container with id c2084fd51b8ee9bc99d3d35edf3b0d89970efa0d39636bac17b14d6e07e53d92 Sep 09 21:02:35.708321 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:02:35.707822 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f2af69c_f46c_412c_ade6_00311e4e59b0.slice/crio-874c42f16a490fa61d22c310df3f9d7ee08d6b02fe09478e4959363c4c171f2a WatchSource:0}: Error finding container 874c42f16a490fa61d22c310df3f9d7ee08d6b02fe09478e4959363c4c171f2a: Status 404 returned error can't find the container with id 874c42f16a490fa61d22c310df3f9d7ee08d6b02fe09478e4959363c4c171f2a Sep 09 21:02:36.088954 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:36.088872 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" event={"ID":"0f2af69c-f46c-412c-ade6-00311e4e59b0","Type":"ContainerStarted","Data":"874c42f16a490fa61d22c310df3f9d7ee08d6b02fe09478e4959363c4c171f2a"} Sep 09 21:02:36.090544 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:36.090510 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xbvrp" event={"ID":"e717cc32-b9ce-46d2-b624-1391699be568","Type":"ContainerStarted","Data":"c8a1aa09b9988b99452e7b9faddc787a3a29d7869a73d6f214fd905e2bb12fc9"} Sep 09 21:02:36.090679 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:36.090565 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:02:36.093660 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:36.093632 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" event={"ID":"2da07b2d-4244-45d8-87e5-ada3a82297fc","Type":"ContainerStarted","Data":"c2084fd51b8ee9bc99d3d35edf3b0d89970efa0d39636bac17b14d6e07e53d92"} Sep 09 21:02:36.111117 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:36.111062 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xbvrp" podStartSLOduration=66.437441917 podStartE2EDuration="1m11.111018594s" totalImagesPullingTime="4.673576677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:36.110283103 +0000 UTC m=+71.971805497" watchObservedRunningTime="2026-09-09 21:02:36.111018594 +0000 UTC m=+71.972540991" Sep 09 21:02:37.051712 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:37.051673 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:37.436351 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:37.436267 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:37.569408 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:37.569374 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:38.028646 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:38.028604 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:38.145097 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:38.145062 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:38.240312 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:38.240267 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:38.894051 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:38.893954 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:39.006388 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.006349 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:39.104239 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.104199 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" event={"ID":"2da07b2d-4244-45d8-87e5-ada3a82297fc","Type":"ContainerStarted","Data":"e1d1a8b80b225d0faf8f83e510d4a6ae8592be461f4643f63dd3b3f2a333a40c"} Sep 09 21:02:39.105866 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.105836 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gj6c4" event={"ID":"253428b3-7b9b-474c-b3eb-f8b8f63b9671","Type":"ContainerStarted","Data":"0610902a789665b4ee8a8bae7728aa93b3d2da97f159bc761649442c0ed09a52"} Sep 09 21:02:39.113540 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.113499 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:39.190483 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.190246 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" podStartSLOduration=2.438544554 podStartE2EDuration="5.190227525s" totalImagesPullingTime="2.751682971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:39.15152718 +0000 UTC m=+75.013049575" watchObservedRunningTime="2026-09-09 21:02:39.190227525 +0000 UTC m=+75.051749910" Sep 09 21:02:39.190823 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.190633 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gj6c4" podStartSLOduration=66.695319286 podStartE2EDuration="1m14.190623224s" totalImagesPullingTime="7.495303938s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:39.190129341 +0000 UTC m=+75.051651739" watchObservedRunningTime="2026-09-09 21:02:39.190623224 +0000 UTC m=+75.052145620" Sep 09 21:02:39.225719 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.225678 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:39.587555 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.587512 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:39.587750 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.587729 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-d99k5"] Sep 09 21:02:39.590760 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.590739 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:39.593211 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.593188 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 09 21:02:39.593428 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.593413 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 09 21:02:39.593566 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.593496 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2hhmg\"" Sep 09 21:02:39.620099 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.620066 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:39.620099 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.620101 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4hxx\" (UniqueName: \"kubernetes.io/projected/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-api-access-s4hxx\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:39.620292 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.620198 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:39.620292 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.620254 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/af7ee458-4682-48c8-90d3-c2c2673ed791-crio-socket\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:39.620292 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.620285 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/af7ee458-4682-48c8-90d3-c2c2673ed791-data-volume\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:39.658097 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.658065 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:39.662959 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.662931 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d99k5"] Sep 09 21:02:39.721575 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.721539 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:39.721575 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.721578 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4hxx\" (UniqueName: \"kubernetes.io/projected/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-api-access-s4hxx\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:39.721841 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.721709 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:39.721841 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.721751 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/af7ee458-4682-48c8-90d3-c2c2673ed791-crio-socket\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:39.721841 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:39.721711 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:02:39.721841 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.721783 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/af7ee458-4682-48c8-90d3-c2c2673ed791-data-volume\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:39.721841 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:39.721829 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls podName:af7ee458-4682-48c8-90d3-c2c2673ed791 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:40.221811194 +0000 UTC m=+76.083333572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d99k5" (UID: "af7ee458-4682-48c8-90d3-c2c2673ed791") : secret "insights-runtime-extractor-tls" not found Sep 09 21:02:39.722079 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:39.721768 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:02:39.722079 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.721895 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/af7ee458-4682-48c8-90d3-c2c2673ed791-crio-socket\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:39.722079 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:39.721909 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm podName:af7ee458-4682-48c8-90d3-c2c2673ed791 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:40.221890442 +0000 UTC m=+76.083412833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d99k5" (UID: "af7ee458-4682-48c8-90d3-c2c2673ed791") : configmap "kube-rbac-proxy" not found Sep 09 21:02:39.722184 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.722123 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/af7ee458-4682-48c8-90d3-c2c2673ed791-data-volume\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:39.733352 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:39.733323 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4hxx\" (UniqueName: \"kubernetes.io/projected/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-api-access-s4hxx\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:40.225637 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:40.225519 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:40.226134 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:40.225797 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:02:40.226134 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:40.225892 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm podName:af7ee458-4682-48c8-90d3-c2c2673ed791 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:41.225868916 +0000 UTC m=+77.087391289 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d99k5" (UID: "af7ee458-4682-48c8-90d3-c2c2673ed791") : configmap "kube-rbac-proxy" not found Sep 09 21:02:40.226134 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:40.225926 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:40.226134 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:40.226024 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:02:40.226134 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:40.226104 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls podName:af7ee458-4682-48c8-90d3-c2c2673ed791 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:41.226087472 +0000 UTC m=+77.087609859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d99k5" (UID: "af7ee458-4682-48c8-90d3-c2c2673ed791") : secret "insights-runtime-extractor-tls" not found Sep 09 21:02:41.112373 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:41.112289 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" event={"ID":"0f2af69c-f46c-412c-ade6-00311e4e59b0","Type":"ContainerStarted","Data":"5fb0d267b87d75bc4f4a2a5f146bcc6282588ce7c27e4af4757c4effa7f8c85c"} Sep 09 21:02:41.132424 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:41.132370 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" podStartSLOduration=2.114362934 podStartE2EDuration="7.132349941s" totalImagesPullingTime="5.017987007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:41.131004616 +0000 UTC m=+76.992527012" watchObservedRunningTime="2026-09-09 21:02:41.132349941 +0000 UTC m=+76.993872337" Sep 09 21:02:41.233764 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:41.233730 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:41.234239 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:41.233854 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:02:41.234239 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:41.233974 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls podName:af7ee458-4682-48c8-90d3-c2c2673ed791 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:43.233955129 +0000 UTC m=+79.095477518 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d99k5" (UID: "af7ee458-4682-48c8-90d3-c2c2673ed791") : secret "insights-runtime-extractor-tls" not found Sep 09 21:02:41.234239 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:41.234002 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:41.234239 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:41.234112 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:02:41.234239 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:41.234160 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm podName:af7ee458-4682-48c8-90d3-c2c2673ed791 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:43.234147962 +0000 UTC m=+79.095670353 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d99k5" (UID: "af7ee458-4682-48c8-90d3-c2c2673ed791") : configmap "kube-rbac-proxy" not found Sep 09 21:02:42.152690 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:42.152656 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jq8n4_71b2bab2-31a6-4b4b-9f09-44ffbfd210c6/dns-node-resolver/0.log" Sep 09 21:02:42.391200 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:42.391166 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:42.454124 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:42.454083 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:42.827513 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:42.827422 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:43.052979 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:43.052944 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:43.151091 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:43.151006 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:43.249449 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:43.249409 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:43.249610 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:43.249499 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:43.249610 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:43.249532 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:02:43.249610 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:43.249603 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm podName:af7ee458-4682-48c8-90d3-c2c2673ed791 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:47.249582777 +0000 UTC m=+83.111105161 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d99k5" (UID: "af7ee458-4682-48c8-90d3-c2c2673ed791") : configmap "kube-rbac-proxy" not found Sep 09 21:02:43.249610 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:43.249604 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:02:43.249747 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:43.249639 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls podName:af7ee458-4682-48c8-90d3-c2c2673ed791 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:47.249630688 +0000 UTC m=+83.111153061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d99k5" (UID: "af7ee458-4682-48c8-90d3-c2c2673ed791") : secret "insights-runtime-extractor-tls" not found Sep 09 21:02:43.538554 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:43.538523 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sdc89_76b95cc0-0a15-4c20-bd29-463861438b51/node-ca/0.log" Sep 09 21:02:44.337682 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.337652 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nv9wb_588a22bb-35b2-43cc-86fa-50afff61eb82/serve-healthcheck-canary/0.log" Sep 09 21:02:44.707430 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.707392 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-vxzlh"] Sep 09 21:02:44.710553 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.710532 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-vxzlh" Sep 09 21:02:44.713444 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.713425 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-thphq\"" Sep 09 21:02:44.713660 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.713646 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 09 21:02:44.714429 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.714415 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 09 21:02:44.714521 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.714434 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 09 21:02:44.714521 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.714434 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 09 21:02:44.723351 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.723331 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-vxzlh"] Sep 09 21:02:44.752632 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.752605 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:44.761694 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.761671 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/771cfeea-8e7a-42dc-aedf-2caffe2c4d30-signing-cabundle\") pod \"service-ca-75d99c484-vxzlh\" (UID: \"771cfeea-8e7a-42dc-aedf-2caffe2c4d30\") " pod="openshift-service-ca/service-ca-75d99c484-vxzlh" Sep 09 21:02:44.761783 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.761716 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqkxw\" (UniqueName: \"kubernetes.io/projected/771cfeea-8e7a-42dc-aedf-2caffe2c4d30-kube-api-access-tqkxw\") pod \"service-ca-75d99c484-vxzlh\" (UID: \"771cfeea-8e7a-42dc-aedf-2caffe2c4d30\") " pod="openshift-service-ca/service-ca-75d99c484-vxzlh" Sep 09 21:02:44.761783 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.761745 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/771cfeea-8e7a-42dc-aedf-2caffe2c4d30-signing-key\") pod \"service-ca-75d99c484-vxzlh\" (UID: \"771cfeea-8e7a-42dc-aedf-2caffe2c4d30\") " pod="openshift-service-ca/service-ca-75d99c484-vxzlh" Sep 09 21:02:44.862139 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.862110 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/771cfeea-8e7a-42dc-aedf-2caffe2c4d30-signing-cabundle\") pod \"service-ca-75d99c484-vxzlh\" (UID: \"771cfeea-8e7a-42dc-aedf-2caffe2c4d30\") " pod="openshift-service-ca/service-ca-75d99c484-vxzlh" Sep 09 21:02:44.862299 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.862168 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqkxw\" (UniqueName: \"kubernetes.io/projected/771cfeea-8e7a-42dc-aedf-2caffe2c4d30-kube-api-access-tqkxw\") pod \"service-ca-75d99c484-vxzlh\" (UID: \"771cfeea-8e7a-42dc-aedf-2caffe2c4d30\") " pod="openshift-service-ca/service-ca-75d99c484-vxzlh" Sep 09 21:02:44.862299 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.862201 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/771cfeea-8e7a-42dc-aedf-2caffe2c4d30-signing-key\") pod \"service-ca-75d99c484-vxzlh\" (UID: \"771cfeea-8e7a-42dc-aedf-2caffe2c4d30\") " pod="openshift-service-ca/service-ca-75d99c484-vxzlh" Sep 09 21:02:44.862700 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.862679 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/771cfeea-8e7a-42dc-aedf-2caffe2c4d30-signing-cabundle\") pod \"service-ca-75d99c484-vxzlh\" (UID: \"771cfeea-8e7a-42dc-aedf-2caffe2c4d30\") " pod="openshift-service-ca/service-ca-75d99c484-vxzlh" Sep 09 21:02:44.864620 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.864568 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/771cfeea-8e7a-42dc-aedf-2caffe2c4d30-signing-key\") pod \"service-ca-75d99c484-vxzlh\" (UID: \"771cfeea-8e7a-42dc-aedf-2caffe2c4d30\") " pod="openshift-service-ca/service-ca-75d99c484-vxzlh" Sep 09 21:02:44.871656 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:44.871629 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqkxw\" (UniqueName: \"kubernetes.io/projected/771cfeea-8e7a-42dc-aedf-2caffe2c4d30-kube-api-access-tqkxw\") pod \"service-ca-75d99c484-vxzlh\" (UID: \"771cfeea-8e7a-42dc-aedf-2caffe2c4d30\") " pod="openshift-service-ca/service-ca-75d99c484-vxzlh" Sep 09 21:02:45.019540 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:45.019451 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-vxzlh" Sep 09 21:02:45.297602 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:45.297524 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-vxzlh"] Sep 09 21:02:45.301124 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:02:45.301102 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod771cfeea_8e7a_42dc_aedf_2caffe2c4d30.slice/crio-264cabaecc18b1f83d1384506e19b4fd58d808ef9672175152ed4e48dc37fc89 WatchSource:0}: Error finding container 264cabaecc18b1f83d1384506e19b4fd58d808ef9672175152ed4e48dc37fc89: Status 404 returned error can't find the container with id 264cabaecc18b1f83d1384506e19b4fd58d808ef9672175152ed4e48dc37fc89 Sep 09 21:02:45.827197 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:45.827159 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:46.131275 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:46.131181 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-vxzlh" event={"ID":"771cfeea-8e7a-42dc-aedf-2caffe2c4d30","Type":"ContainerStarted","Data":"992014da11a1a1d65cebf318ea31e6e4c27ebafd0aa0ddd402f397b17d61174f"} Sep 09 21:02:46.131275 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:46.131273 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-vxzlh" event={"ID":"771cfeea-8e7a-42dc-aedf-2caffe2c4d30","Type":"ContainerStarted","Data":"264cabaecc18b1f83d1384506e19b4fd58d808ef9672175152ed4e48dc37fc89"} Sep 09 21:02:46.153485 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:46.153434 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-vxzlh" podStartSLOduration=2.153420578 podStartE2EDuration="2.153420578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:46.151866186 +0000 UTC m=+82.013388592" watchObservedRunningTime="2026-09-09 21:02:46.153420578 +0000 UTC m=+82.014942973" Sep 09 21:02:47.280970 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:47.280826 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:47.280970 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:47.280916 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:47.281359 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:47.280985 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:02:47.281359 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:47.280985 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:02:47.281359 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:47.281086 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls podName:af7ee458-4682-48c8-90d3-c2c2673ed791 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:55.281064223 +0000 UTC m=+91.142586607 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d99k5" (UID: "af7ee458-4682-48c8-90d3-c2c2673ed791") : secret "insights-runtime-extractor-tls" not found Sep 09 21:02:47.281359 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:47.281103 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm podName:af7ee458-4682-48c8-90d3-c2c2673ed791 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:55.281095433 +0000 UTC m=+91.142617806 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d99k5" (UID: "af7ee458-4682-48c8-90d3-c2c2673ed791") : configmap "kube-rbac-proxy" not found Sep 09 21:02:47.514660 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:47.514624 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:47.701132 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:47.701100 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:47.799533 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:47.799481 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:47.894219 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:47.894185 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:48.135074 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:48.134969 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:49.346649 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:49.346614 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:49.608478 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:49.608394 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:49.748521 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:49.748489 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:49.950079 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:49.950053 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:50.200506 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:50.200428 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:51.157844 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:51.157816 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:51.221688 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:51.221658 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:51.335786 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:51.335734 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:51.529970 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:51.529940 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:51.929443 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:51.929352 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:52.030163 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:52.030130 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:52.132096 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:52.132061 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:52.350370 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:52.350323 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:52.730166 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:52.730127 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:53.534283 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:53.534245 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:53.731167 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:53.731132 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:53.802521 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:53.802441 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:53.981984 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:53.981935 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:54.361761 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:54.361727 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:54.750329 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:54.750301 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:55.134707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:55.134627 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:55.347134 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:55.347074 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:55.347134 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:55.347140 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:55.347400 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:55.347239 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:02:55.347400 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:02:55.347320 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls podName:af7ee458-4682-48c8-90d3-c2c2673ed791 nodeName:}" failed. No retries permitted until 2026-09-09 21:03:11.347301468 +0000 UTC m=+107.208823865 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d99k5" (UID: "af7ee458-4682-48c8-90d3-c2c2673ed791") : secret "insights-runtime-extractor-tls" not found Sep 09 21:02:55.347756 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:55.347732 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af7ee458-4682-48c8-90d3-c2c2673ed791-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:02:55.735034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:55.735001 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:56.361434 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:56.361402 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:56.535573 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:56.535541 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:56.953208 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:56.953171 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:57.158256 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:57.158205 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:57.736460 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:57.736430 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:57.965096 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:57.965062 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:58.136142 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:02:58.136064 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:00.565118 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:00.565083 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:00.666172 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:00.666136 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:00.769907 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:00.769879 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:00.868621 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:00.868520 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:01.044169 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:01.044131 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:01.125476 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:01.125387 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:01.245103 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:01.245068 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:01.346361 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:01.346332 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:02.472711 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:02.472678 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:02.560464 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:02.560436 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:02.660736 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:02.660706 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:02.892627 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:02.892540 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:03.413304 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:03.413252 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:03:03.413495 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:03.413334 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:03:03.415668 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:03.415634 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f73f9bb9-8de7-4211-a99d-758f7c2721b5-metrics-tls\") pod \"dns-default-5j4zn\" (UID: \"f73f9bb9-8de7-4211-a99d-758f7c2721b5\") " pod="openshift-dns/dns-default-5j4zn" Sep 09 21:03:03.415778 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:03.415683 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls\") pod \"image-registry-777dbd79f8-87p9r\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:03:03.646574 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:03.646490 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:03.692579 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:03.692556 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xcsfw\"" Sep 09 21:03:03.698879 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:03.698862 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:03:03.716259 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:03.716229 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9vjrs\"" Sep 09 21:03:03.722242 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:03.722224 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5j4zn" Sep 09 21:03:04.000631 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:04.000591 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-777dbd79f8-87p9r"] Sep 09 21:03:04.004248 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:03:04.004223 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfabe4da8_a1c9_4e4b_aff7_65756b9de477.slice/crio-1eb666b8253db336f6256950fc8f3cc6b2f99f64fffc12b19e15d23059692371 WatchSource:0}: Error finding container 1eb666b8253db336f6256950fc8f3cc6b2f99f64fffc12b19e15d23059692371: Status 404 returned error can't find the container with id 1eb666b8253db336f6256950fc8f3cc6b2f99f64fffc12b19e15d23059692371 Sep 09 21:03:04.019460 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:04.019436 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5j4zn"] Sep 09 21:03:04.190609 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:04.190565 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" event={"ID":"fabe4da8-a1c9-4e4b-aff7-65756b9de477","Type":"ContainerStarted","Data":"714e12839c208889555d327a0298a4f4ea26bf3c8e937c12dd94383d7e94caf7"} Sep 09 21:03:04.190816 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:04.190643 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:03:04.190816 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:04.190671 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" event={"ID":"fabe4da8-a1c9-4e4b-aff7-65756b9de477","Type":"ContainerStarted","Data":"1eb666b8253db336f6256950fc8f3cc6b2f99f64fffc12b19e15d23059692371"} Sep 09 21:03:04.191766 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:04.191732 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5j4zn" event={"ID":"f73f9bb9-8de7-4211-a99d-758f7c2721b5","Type":"ContainerStarted","Data":"01303bec5aec1c72f76eb75dd8c82727bfada07feb524d7f611697e81b8b1a2d"} Sep 09 21:03:04.218917 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:04.218863 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" podStartSLOduration=98.218846378 podStartE2EDuration="1m38.218846378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:04.21859503 +0000 UTC m=+100.080117424" watchObservedRunningTime="2026-09-09 21:03:04.218846378 +0000 UTC m=+100.080368773" Sep 09 21:03:06.200822 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:06.200790 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5j4zn" event={"ID":"f73f9bb9-8de7-4211-a99d-758f7c2721b5","Type":"ContainerStarted","Data":"ebd77ef036b014a1fb6454ffd8c2d3215cdfde722893fc86b6fb9c202eb50e78"} Sep 09 21:03:07.099476 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:07.099446 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xbvrp" Sep 09 21:03:07.205978 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:07.205944 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5j4zn" event={"ID":"f73f9bb9-8de7-4211-a99d-758f7c2721b5","Type":"ContainerStarted","Data":"d1b0b706391b5b5b97f8750aa89f311f565e4f78ee89986aba80809d83f8b927"} Sep 09 21:03:07.206454 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:07.206088 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5j4zn" Sep 09 21:03:07.234580 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:07.234537 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5j4zn" podStartSLOduration=66.161933958 podStartE2EDuration="1m8.234521178s" totalImagesPullingTime="2.07258722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:07.233559625 +0000 UTC m=+103.095082021" watchObservedRunningTime="2026-09-09 21:03:07.234521178 +0000 UTC m=+103.096043627" Sep 09 21:03:09.599936 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:09.599900 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:09.700157 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:09.700125 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:09.910328 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:09.910245 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:10.024092 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:10.024034 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:11.291438 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:11.291411 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:11.377741 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:11.377703 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:03:11.379949 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:11.379926 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af7ee458-4682-48c8-90d3-c2c2673ed791-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d99k5\" (UID: \"af7ee458-4682-48c8-90d3-c2c2673ed791\") " pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:03:11.400134 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:11.400100 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d99k5" Sep 09 21:03:11.727559 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:11.727529 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d99k5"] Sep 09 21:03:11.731066 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:03:11.731028 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf7ee458_4682_48c8_90d3_c2c2673ed791.slice/crio-a113d3b8b96373ffc4175d00d536f2c9fc6ac2c8e429b2872db4c973466a7b8e WatchSource:0}: Error finding container a113d3b8b96373ffc4175d00d536f2c9fc6ac2c8e429b2872db4c973466a7b8e: Status 404 returned error can't find the container with id a113d3b8b96373ffc4175d00d536f2c9fc6ac2c8e429b2872db4c973466a7b8e Sep 09 21:03:12.223258 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:12.223224 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d99k5" event={"ID":"af7ee458-4682-48c8-90d3-c2c2673ed791","Type":"ContainerStarted","Data":"30193dc25c263ee4bda06d8224d57a051e671dad09ed3f8984ee4112392a2836"} Sep 09 21:03:12.223422 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:12.223280 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d99k5" event={"ID":"af7ee458-4682-48c8-90d3-c2c2673ed791","Type":"ContainerStarted","Data":"a113d3b8b96373ffc4175d00d536f2c9fc6ac2c8e429b2872db4c973466a7b8e"} Sep 09 21:03:14.234317 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:14.234278 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d99k5" event={"ID":"af7ee458-4682-48c8-90d3-c2c2673ed791","Type":"ContainerStarted","Data":"dfe4b20df117c2625e4f87427a0b8029d19c88a7e18fb9c2a410360023966ba5"} Sep 09 21:03:14.674076 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:14.673982 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:14.773163 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:14.773125 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:15.239852 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:15.239815 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d99k5" event={"ID":"af7ee458-4682-48c8-90d3-c2c2673ed791","Type":"ContainerStarted","Data":"671919cc5eb0dcd6849aa8ca65c7e6ee828f6a845b84a031b00c5ed8ebf55180"} Sep 09 21:03:15.279397 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:15.279307 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-d99k5" podStartSLOduration=33.25760427 podStartE2EDuration="36.279291796s" totalImagesPullingTime="3.021687526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:02:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:15.278401278 +0000 UTC m=+111.139923672" watchObservedRunningTime="2026-09-09 21:03:15.279291796 +0000 UTC m=+111.140814211" Sep 09 21:03:15.294138 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:15.294110 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:15.378410 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:15.378380 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:17.211399 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:17.211372 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5j4zn" Sep 09 21:03:20.292244 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:20.292209 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:20.392591 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:20.392560 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:20.755236 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:20.755202 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:20.855207 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:20.855172 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:21.005517 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.005431 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:21.024445 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.024412 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-6jqtn"] Sep 09 21:03:21.026396 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.026375 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-6jqtn" Sep 09 21:03:21.027591 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.027563 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh"] Sep 09 21:03:21.029369 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.029349 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 09 21:03:21.029481 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.029394 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 09 21:03:21.029481 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.029430 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8gbdv\"" Sep 09 21:03:21.034801 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.034773 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.037194 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.037170 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 09 21:03:21.037194 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.037190 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 09 21:03:21.037353 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.037196 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 09 21:03:21.037475 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.037450 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 09 21:03:21.037575 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.037500 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 09 21:03:21.037575 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.037502 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 09 21:03:21.038158 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.038139 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 09 21:03:21.056840 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.056813 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh"] Sep 09 21:03:21.091472 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.091440 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-6jqtn"] Sep 09 21:03:21.150599 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.150569 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.150777 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.150620 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkpxk\" (UniqueName: \"kubernetes.io/projected/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-kube-api-access-kkpxk\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.150777 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.150646 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-hub\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.150777 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.150673 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a130f42-a944-429a-b888-56792ed2d998-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-6jqtn\" (UID: \"9a130f42-a944-429a-b888-56792ed2d998\") " pod="openshift-network-console/networking-console-plugin-79977f9844-6jqtn" Sep 09 21:03:21.150777 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.150694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.150777 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.150725 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.150970 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.150801 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-ca\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.150970 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.150823 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9a130f42-a944-429a-b888-56792ed2d998-nginx-conf\") pod \"networking-console-plugin-79977f9844-6jqtn\" (UID: \"9a130f42-a944-429a-b888-56792ed2d998\") " pod="openshift-network-console/networking-console-plugin-79977f9844-6jqtn" Sep 09 21:03:21.251952 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.251915 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-ca\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.251952 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.251953 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9a130f42-a944-429a-b888-56792ed2d998-nginx-conf\") pod \"networking-console-plugin-79977f9844-6jqtn\" (UID: \"9a130f42-a944-429a-b888-56792ed2d998\") " pod="openshift-network-console/networking-console-plugin-79977f9844-6jqtn" Sep 09 21:03:21.252235 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.251979 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.252235 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.252159 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkpxk\" (UniqueName: \"kubernetes.io/projected/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-kube-api-access-kkpxk\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.252235 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.252200 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-hub\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.252390 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.252294 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a130f42-a944-429a-b888-56792ed2d998-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-6jqtn\" (UID: \"9a130f42-a944-429a-b888-56792ed2d998\") " pod="openshift-network-console/networking-console-plugin-79977f9844-6jqtn" Sep 09 21:03:21.252390 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.252336 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.252390 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.252385 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.252821 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.252794 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9a130f42-a944-429a-b888-56792ed2d998-nginx-conf\") pod \"networking-console-plugin-79977f9844-6jqtn\" (UID: \"9a130f42-a944-429a-b888-56792ed2d998\") " pod="openshift-network-console/networking-console-plugin-79977f9844-6jqtn" Sep 09 21:03:21.253159 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.253125 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.254720 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.254696 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.254866 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.254809 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a130f42-a944-429a-b888-56792ed2d998-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-6jqtn\" (UID: \"9a130f42-a944-429a-b888-56792ed2d998\") " pod="openshift-network-console/networking-console-plugin-79977f9844-6jqtn" Sep 09 21:03:21.254939 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.254903 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.255096 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.255079 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-hub\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.255168 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.255148 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-ca\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.261809 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.261753 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkpxk\" (UniqueName: \"kubernetes.io/projected/63b6652f-74d6-46b1-ae5f-82aeeae7eaba-kube-api-access-kkpxk\") pod \"cluster-proxy-proxy-agent-7bc8446c84-8p8hh\" (UID: \"63b6652f-74d6-46b1-ae5f-82aeeae7eaba\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.337515 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.337478 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-6jqtn" Sep 09 21:03:21.355431 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.355399 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" Sep 09 21:03:21.423290 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.423235 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:21.620851 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.620768 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-6jqtn"] Sep 09 21:03:21.625632 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:03:21.625603 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a130f42_a944_429a_b888_56792ed2d998.slice/crio-d91cab5dccb25e474bdeb35105fd4110861c60254eff8d401cbe0a21ed37caf7 WatchSource:0}: Error finding container d91cab5dccb25e474bdeb35105fd4110861c60254eff8d401cbe0a21ed37caf7: Status 404 returned error can't find the container with id d91cab5dccb25e474bdeb35105fd4110861c60254eff8d401cbe0a21ed37caf7 Sep 09 21:03:21.667209 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.667177 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh"] Sep 09 21:03:21.671468 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:03:21.671445 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63b6652f_74d6_46b1_ae5f_82aeeae7eaba.slice/crio-2caf9ed0c2c81857080f080a6354cc0def895689c59baf18c7d0e49f4b783dc1 WatchSource:0}: Error finding container 2caf9ed0c2c81857080f080a6354cc0def895689c59baf18c7d0e49f4b783dc1: Status 404 returned error can't find the container with id 2caf9ed0c2c81857080f080a6354cc0def895689c59baf18c7d0e49f4b783dc1 Sep 09 21:03:21.691369 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:21.691342 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:22.265121 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:22.265078 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" event={"ID":"63b6652f-74d6-46b1-ae5f-82aeeae7eaba","Type":"ContainerStarted","Data":"2caf9ed0c2c81857080f080a6354cc0def895689c59baf18c7d0e49f4b783dc1"} Sep 09 21:03:22.266182 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:22.266151 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-6jqtn" event={"ID":"9a130f42-a944-429a-b888-56792ed2d998","Type":"ContainerStarted","Data":"d91cab5dccb25e474bdeb35105fd4110861c60254eff8d401cbe0a21ed37caf7"} Sep 09 21:03:23.704892 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:23.704839 2560 patch_prober.go:28] interesting pod/image-registry-777dbd79f8-87p9r container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:03:23.705359 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:23.704947 2560 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" podUID="fabe4da8-a1c9-4e4b-aff7-65756b9de477" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:03:24.275191 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:24.275156 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-6jqtn" event={"ID":"9a130f42-a944-429a-b888-56792ed2d998","Type":"ContainerStarted","Data":"0c5c6996340db39d4d711527225148f0e0775daf22cf24b5b53e3e6e06e68d64"} Sep 09 21:03:24.299548 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:24.299480 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-6jqtn" podStartSLOduration=0.775696207 podStartE2EDuration="3.299460887s" totalImagesPullingTime="2.52376468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:24.298780116 +0000 UTC m=+120.160302512" watchObservedRunningTime="2026-09-09 21:03:24.299460887 +0000 UTC m=+120.160983285" Sep 09 21:03:24.436226 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:24.436129 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:24.501203 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:24.501160 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:25.034417 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:25.034379 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:25.103910 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:25.103880 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:25.204884 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:25.203784 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:25.205153 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:25.205116 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:03:25.487113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:25.487082 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:25.587200 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:25.587168 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:26.284246 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:26.284212 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" event={"ID":"63b6652f-74d6-46b1-ae5f-82aeeae7eaba","Type":"ContainerStarted","Data":"6d1ef5115860573e36132de661d76abea43a3fabf0de140b82b8aaadc4ea233c"} Sep 09 21:03:26.284646 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:26.284257 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" event={"ID":"63b6652f-74d6-46b1-ae5f-82aeeae7eaba","Type":"ContainerStarted","Data":"2289ba8f23d4a2c48392560ac226a2895f27ca75641e76ddd2f2b4cd62454dd8"} Sep 09 21:03:26.284646 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:26.284273 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" event={"ID":"63b6652f-74d6-46b1-ae5f-82aeeae7eaba","Type":"ContainerStarted","Data":"a31e904edb546ec361c8e6922b0678e683148160b2e2e5bf1ca8efe12d8d32c7"} Sep 09 21:03:26.317566 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:26.317511 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc8446c84-8p8hh" podStartSLOduration=1.551595179 podStartE2EDuration="5.317495926s" totalImagesPullingTime="3.765900747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:26.316497179 +0000 UTC m=+122.178019574" watchObservedRunningTime="2026-09-09 21:03:26.317495926 +0000 UTC m=+122.179018302" Sep 09 21:03:27.679183 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:27.679146 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:27.779173 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:27.779138 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:28.231280 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:28.231251 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:28.300157 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:28.300127 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:30.126547 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.126507 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:30.340920 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.340879 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:30.380447 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.380358 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-g4nlp"] Sep 09 21:03:30.383020 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.383002 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.386397 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.386376 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 09 21:03:30.386883 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.386866 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-brllz\"" Sep 09 21:03:30.388111 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.388093 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 09 21:03:30.388400 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.388374 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 09 21:03:30.388824 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.388806 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 09 21:03:30.388908 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.388864 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 09 21:03:30.388968 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.388926 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 09 21:03:30.416846 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.416813 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-wtmp\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.416846 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.416849 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-sys\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.417140 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.416880 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-metrics-client-ca\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.417140 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.416923 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-root\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.417140 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.416942 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.417140 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.416987 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.417140 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.417018 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-tls\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.417140 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.417056 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-textfile\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.417140 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.417078 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nvj6\" (UniqueName: \"kubernetes.io/projected/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-kube-api-access-8nvj6\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.418241 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.418198 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:30.512157 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.512096 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:30.517666 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.517645 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-root\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.517748 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.517686 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.517809 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.517759 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-root\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.517858 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.517816 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.517913 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.517873 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-tls\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.517960 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.517908 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-textfile\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.517960 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.517937 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nvj6\" (UniqueName: \"kubernetes.io/projected/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-kube-api-access-8nvj6\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.518077 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.518009 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-wtmp\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.518077 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.518036 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-sys\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.518601 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.518192 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-wtmp\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.518601 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.518227 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-metrics-client-ca\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.518601 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.518353 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-sys\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.518601 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.518450 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.518601 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.518524 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-textfile\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.518914 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.518752 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-metrics-client-ca\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.520441 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.520420 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-tls\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.520680 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.520661 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.527165 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.527138 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nvj6\" (UniqueName: \"kubernetes.io/projected/87c8eb5c-6363-4821-adc5-cc6eb11a91c4-kube-api-access-8nvj6\") pod \"node-exporter-g4nlp\" (UID: \"87c8eb5c-6363-4821-adc5-cc6eb11a91c4\") " pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.623573 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.623542 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:30.692677 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.692578 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g4nlp" Sep 09 21:03:30.703338 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:03:30.703311 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c8eb5c_6363_4821_adc5_cc6eb11a91c4.slice/crio-031c7936d653e67195349a1dcb4d63243422c08215c817b35ee671f82c07a6f4 WatchSource:0}: Error finding container 031c7936d653e67195349a1dcb4d63243422c08215c817b35ee671f82c07a6f4: Status 404 returned error can't find the container with id 031c7936d653e67195349a1dcb4d63243422c08215c817b35ee671f82c07a6f4 Sep 09 21:03:30.785302 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.785265 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:30.940353 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:30.940318 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:31.041229 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:31.041198 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:31.304218 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:31.304124 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g4nlp" event={"ID":"87c8eb5c-6363-4821-adc5-cc6eb11a91c4","Type":"ContainerStarted","Data":"031c7936d653e67195349a1dcb4d63243422c08215c817b35ee671f82c07a6f4"} Sep 09 21:03:32.311749 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:32.311716 2560 generic.go:356] "Generic (PLEG): container finished" podID="87c8eb5c-6363-4821-adc5-cc6eb11a91c4" containerID="39b3a31586f0dcf3b295da7e37452978c8bfccad7835a4d2caf5d2d3492b1739" exitCode=0 Sep 09 21:03:32.312169 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:32.311784 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g4nlp" event={"ID":"87c8eb5c-6363-4821-adc5-cc6eb11a91c4","Type":"ContainerDied","Data":"39b3a31586f0dcf3b295da7e37452978c8bfccad7835a4d2caf5d2d3492b1739"} Sep 09 21:03:32.342506 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:32.342477 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:32.444444 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:32.444415 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:33.317536 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:33.317501 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g4nlp" event={"ID":"87c8eb5c-6363-4821-adc5-cc6eb11a91c4","Type":"ContainerStarted","Data":"834188e97970bbf7f365f3e643deb308224e95a67f7abf8b246f0e66243db7ca"} Sep 09 21:03:33.318016 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:33.317550 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g4nlp" event={"ID":"87c8eb5c-6363-4821-adc5-cc6eb11a91c4","Type":"ContainerStarted","Data":"786758c099d153b7c14978d90c609a2e03cd3ac49a84da22e4bf3dde95cd5ec7"} Sep 09 21:03:33.395462 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:33.395396 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-g4nlp" podStartSLOduration=2.420683054 podStartE2EDuration="3.39537786s" totalImagesPullingTime="911.435987ms" totalInitContainerRuntime="63.258819ms" isStatefulPod=true podCreationTimestamp="2026-09-09 21:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:33.393153698 +0000 UTC m=+129.254676093" watchObservedRunningTime="2026-09-09 21:03:33.39537786 +0000 UTC m=+129.256900251" Sep 09 21:03:34.450352 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:34.450310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs\") pod \"network-metrics-daemon-mxztj\" (UID: \"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41\") " pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:03:34.452730 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:34.452703 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41-metrics-certs\") pod \"network-metrics-daemon-mxztj\" (UID: \"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41\") " pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:03:34.588822 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:34.588786 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rgvxk\"" Sep 09 21:03:34.596113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:34.596090 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxztj" Sep 09 21:03:34.874789 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:34.874707 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mxztj"] Sep 09 21:03:34.877745 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:03:34.877705 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc50dfcc_7e7f_48e3_bd05_9feec3e9bc41.slice/crio-6351ba1cf3f202becc16c6ee3285da8d059eff2d513ed3153dbf7c900f1b9229 WatchSource:0}: Error finding container 6351ba1cf3f202becc16c6ee3285da8d059eff2d513ed3153dbf7c900f1b9229: Status 404 returned error can't find the container with id 6351ba1cf3f202becc16c6ee3285da8d059eff2d513ed3153dbf7c900f1b9229 Sep 09 21:03:35.147093 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:35.147007 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:35.216842 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:35.216816 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:35.325219 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:35.325184 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mxztj" event={"ID":"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41","Type":"ContainerStarted","Data":"6351ba1cf3f202becc16c6ee3285da8d059eff2d513ed3153dbf7c900f1b9229"} Sep 09 21:03:35.370874 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:35.370843 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:35.471728 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:35.471698 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:35.571248 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:35.571213 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:35.929492 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:35.929423 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:36.031098 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:36.031064 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:36.737285 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:36.737254 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:36.841202 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:36.841161 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:36.943958 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:36.943925 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:37.013013 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:37.012902 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:37.334394 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:37.334301 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mxztj" event={"ID":"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41","Type":"ContainerStarted","Data":"53a75ed514f0f02c496935c357d486ee8688197fcd9e210a12f787f914f2ad6d"} Sep 09 21:03:37.334394 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:37.334360 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mxztj" event={"ID":"bc50dfcc-7e7f-48e3-bd05-9feec3e9bc41","Type":"ContainerStarted","Data":"11c959bc955e95a9957df27dddb8aab64465441d39481fc0e31a6db56710a024"} Sep 09 21:03:37.355597 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:37.355539 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mxztj" podStartSLOduration=130.92394737 podStartE2EDuration="2m12.355523712s" totalImagesPullingTime="1.431576342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:37.353782574 +0000 UTC m=+133.215304970" watchObservedRunningTime="2026-09-09 21:03:37.355523712 +0000 UTC m=+133.217046107" Sep 09 21:03:37.469617 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:37.469585 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:37.572788 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:37.572755 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:39.463812 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:39.463778 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:39.564264 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:39.564230 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:43.163011 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:43.162975 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-777dbd79f8-87p9r"] Sep 09 21:03:43.296677 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:43.296634 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:43.362759 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:43.362732 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:43.443773 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:43.443642 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:43.545021 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:43.544989 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:45.111878 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:45.111843 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:45.212562 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:45.212524 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:46.111230 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:46.111196 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:46.180201 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:46.180148 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:47.348221 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:47.348162 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:47.415160 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:47.415096 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:48.472808 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:48.472777 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:48.568157 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:48.568107 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:49.374833 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:49.374799 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:49.541812 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:49.541784 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:49.785933 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:49.785860 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:49.929757 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:49.929724 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:50.030369 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:50.030341 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:50.156200 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:50.156096 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:50.261457 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:50.261425 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:50.379109 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:50.379075 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:50.743492 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:50.743444 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:50.845096 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:50.845054 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:51.426364 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:51.426330 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:51.517141 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:51.517105 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:53.288323 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:53.288262 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:03:53.388732 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:03:53.388641 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:02.338572 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:02.338524 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:02.440101 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:02.440057 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:02.958434 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:02.958379 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:03.026115 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:03.026076 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:03.118963 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:03.118930 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:03.219079 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:03.218961 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:03.904318 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:03.904265 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:04.003897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:04.003867 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:04.427242 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:04.427159 2560 generic.go:356] "Generic (PLEG): container finished" podID="2da07b2d-4244-45d8-87e5-ada3a82297fc" containerID="e1d1a8b80b225d0faf8f83e510d4a6ae8592be461f4643f63dd3b3f2a333a40c" exitCode=0 Sep 09 21:04:04.427242 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:04.427203 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" event={"ID":"2da07b2d-4244-45d8-87e5-ada3a82297fc","Type":"ContainerDied","Data":"e1d1a8b80b225d0faf8f83e510d4a6ae8592be461f4643f63dd3b3f2a333a40c"} Sep 09 21:04:04.427525 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:04.427511 2560 scope.go:117] "RemoveContainer" containerID="e1d1a8b80b225d0faf8f83e510d4a6ae8592be461f4643f63dd3b3f2a333a40c" Sep 09 21:04:04.693541 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:04.693459 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:04.793232 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:04.793194 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:05.174835 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:05.174810 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nv9wb_588a22bb-35b2-43cc-86fa-50afff61eb82/serve-healthcheck-canary/0.log" Sep 09 21:04:05.432199 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:05.432115 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-rrrx7" event={"ID":"2da07b2d-4244-45d8-87e5-ada3a82297fc","Type":"ContainerStarted","Data":"83603638e531e54facde9c077b9fa1ac5bc6c7e50e2db8b6f36dd241d87b38bb"} Sep 09 21:04:08.186035 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.185987 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" podUID="fabe4da8-a1c9-4e4b-aff7-65756b9de477" containerName="registry" containerID="cri-o://714e12839c208889555d327a0298a4f4ea26bf3c8e937c12dd94383d7e94caf7" gracePeriod=30 Sep 09 21:04:08.426227 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.426204 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:04:08.443067 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.442981 2560 generic.go:356] "Generic (PLEG): container finished" podID="fabe4da8-a1c9-4e4b-aff7-65756b9de477" containerID="714e12839c208889555d327a0298a4f4ea26bf3c8e937c12dd94383d7e94caf7" exitCode=0 Sep 09 21:04:08.443067 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.443017 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" Sep 09 21:04:08.443219 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.443069 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" event={"ID":"fabe4da8-a1c9-4e4b-aff7-65756b9de477","Type":"ContainerDied","Data":"714e12839c208889555d327a0298a4f4ea26bf3c8e937c12dd94383d7e94caf7"} Sep 09 21:04:08.443219 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.443117 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-777dbd79f8-87p9r" event={"ID":"fabe4da8-a1c9-4e4b-aff7-65756b9de477","Type":"ContainerDied","Data":"1eb666b8253db336f6256950fc8f3cc6b2f99f64fffc12b19e15d23059692371"} Sep 09 21:04:08.443219 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.443144 2560 scope.go:117] "RemoveContainer" containerID="714e12839c208889555d327a0298a4f4ea26bf3c8e937c12dd94383d7e94caf7" Sep 09 21:04:08.450953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.450933 2560 scope.go:117] "RemoveContainer" containerID="714e12839c208889555d327a0298a4f4ea26bf3c8e937c12dd94383d7e94caf7" Sep 09 21:04:08.451266 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:04:08.451247 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"714e12839c208889555d327a0298a4f4ea26bf3c8e937c12dd94383d7e94caf7\": container with ID starting with 714e12839c208889555d327a0298a4f4ea26bf3c8e937c12dd94383d7e94caf7 not found: ID does not exist" containerID="714e12839c208889555d327a0298a4f4ea26bf3c8e937c12dd94383d7e94caf7" Sep 09 21:04:08.451319 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.451280 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"714e12839c208889555d327a0298a4f4ea26bf3c8e937c12dd94383d7e94caf7"} err="failed to get container status \"714e12839c208889555d327a0298a4f4ea26bf3c8e937c12dd94383d7e94caf7\": rpc error: code = NotFound desc = could not find container \"714e12839c208889555d327a0298a4f4ea26bf3c8e937c12dd94383d7e94caf7\": container with ID starting with 714e12839c208889555d327a0298a4f4ea26bf3c8e937c12dd94383d7e94caf7 not found: ID does not exist" Sep 09 21:04:08.511727 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.511689 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k9jr\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-kube-api-access-6k9jr\") pod \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " Sep 09 21:04:08.511894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.511739 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fabe4da8-a1c9-4e4b-aff7-65756b9de477-installation-pull-secrets\") pod \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " Sep 09 21:04:08.511894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.511762 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fabe4da8-a1c9-4e4b-aff7-65756b9de477-image-registry-private-configuration\") pod \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " Sep 09 21:04:08.511894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.511785 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fabe4da8-a1c9-4e4b-aff7-65756b9de477-ca-trust-extracted\") pod \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " Sep 09 21:04:08.511894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.511818 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fabe4da8-a1c9-4e4b-aff7-65756b9de477-trusted-ca\") pod \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " Sep 09 21:04:08.511894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.511836 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-certificates\") pod \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " Sep 09 21:04:08.511894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.511867 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls\") pod \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " Sep 09 21:04:08.511894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.511888 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-bound-sa-token\") pod \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\" (UID: \"fabe4da8-a1c9-4e4b-aff7-65756b9de477\") " Sep 09 21:04:08.512373 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.512311 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fabe4da8-a1c9-4e4b-aff7-65756b9de477-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fabe4da8-a1c9-4e4b-aff7-65756b9de477" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:04:08.512498 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.512419 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fabe4da8-a1c9-4e4b-aff7-65756b9de477" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:04:08.514611 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.514563 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fabe4da8-a1c9-4e4b-aff7-65756b9de477-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fabe4da8-a1c9-4e4b-aff7-65756b9de477" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:04:08.514611 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.514599 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fabe4da8-a1c9-4e4b-aff7-65756b9de477" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:04:08.514805 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.514569 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-kube-api-access-6k9jr" (OuterVolumeSpecName: "kube-api-access-6k9jr") pod "fabe4da8-a1c9-4e4b-aff7-65756b9de477" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477"). InnerVolumeSpecName "kube-api-access-6k9jr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:04:08.514805 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.514636 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fabe4da8-a1c9-4e4b-aff7-65756b9de477" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:04:08.514805 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.514650 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fabe4da8-a1c9-4e4b-aff7-65756b9de477-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fabe4da8-a1c9-4e4b-aff7-65756b9de477" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:04:08.520390 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.520367 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fabe4da8-a1c9-4e4b-aff7-65756b9de477-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fabe4da8-a1c9-4e4b-aff7-65756b9de477" (UID: "fabe4da8-a1c9-4e4b-aff7-65756b9de477"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:04:08.613107 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.613073 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-tls\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:04:08.613107 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.613101 2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-bound-sa-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:04:08.613107 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.613110 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k9jr\" (UniqueName: \"kubernetes.io/projected/fabe4da8-a1c9-4e4b-aff7-65756b9de477-kube-api-access-6k9jr\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:04:08.613356 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.613120 2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fabe4da8-a1c9-4e4b-aff7-65756b9de477-installation-pull-secrets\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:04:08.613356 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.613130 2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fabe4da8-a1c9-4e4b-aff7-65756b9de477-image-registry-private-configuration\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:04:08.613356 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.613138 2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fabe4da8-a1c9-4e4b-aff7-65756b9de477-ca-trust-extracted\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:04:08.613356 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.613147 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fabe4da8-a1c9-4e4b-aff7-65756b9de477-trusted-ca\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:04:08.613356 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.613154 2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fabe4da8-a1c9-4e4b-aff7-65756b9de477-registry-certificates\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:04:08.767635 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.767601 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-777dbd79f8-87p9r"] Sep 09 21:04:08.772289 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.772265 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-777dbd79f8-87p9r"] Sep 09 21:04:08.778820 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:08.778795 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fabe4da8-a1c9-4e4b-aff7-65756b9de477" path="/var/lib/kubelet/pods/fabe4da8-a1c9-4e4b-aff7-65756b9de477/volumes" Sep 09 21:04:12.460878 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:12.460844 2560 generic.go:356] "Generic (PLEG): container finished" podID="0f2af69c-f46c-412c-ade6-00311e4e59b0" containerID="5fb0d267b87d75bc4f4a2a5f146bcc6282588ce7c27e4af4757c4effa7f8c85c" exitCode=0 Sep 09 21:04:12.461262 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:12.460915 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" event={"ID":"0f2af69c-f46c-412c-ade6-00311e4e59b0","Type":"ContainerDied","Data":"5fb0d267b87d75bc4f4a2a5f146bcc6282588ce7c27e4af4757c4effa7f8c85c"} Sep 09 21:04:12.461262 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:12.461250 2560 scope.go:117] "RemoveContainer" containerID="5fb0d267b87d75bc4f4a2a5f146bcc6282588ce7c27e4af4757c4effa7f8c85c" Sep 09 21:04:13.467431 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:13.467392 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6qrdt" event={"ID":"0f2af69c-f46c-412c-ade6-00311e4e59b0","Type":"ContainerStarted","Data":"a1ff225b26d03eeef6843dcf90e158349ac6889fe88abe9e4922c75f3d4aab2c"} Sep 09 21:04:17.352810 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:17.352732 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:17.453771 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:17.453736 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:19.422155 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:19.422119 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:20.257120 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:20.257070 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:20.400736 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:20.400685 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:32.347553 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:32.347514 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:32.447387 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:32.447349 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:39.603364 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:39.603324 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:39.685725 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:04:39.685694 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:05:35.284693 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:05:35.284650 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:05:35.350338 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:05:35.350307 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:05:43.459052 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:05:43.459005 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:05:58.467539 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:05:58.467454 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:06:24.554961 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:06:24.554934 2560 kubelet.go:1648] "Image garbage collection succeeded" Sep 09 21:08:34.593340 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:08:34.593303 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:08:36.288089 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:08:36.288051 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:08:36.655891 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:08:36.655806 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:08:36.796547 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:08:36.796509 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:08:36.899438 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:08:36.899402 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:08:37.002379 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:08:37.002346 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:08:40.017447 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:08:40.017410 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:09:39.714989 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:09:39.714954 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:09:39.813345 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:09:39.813284 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:09:49.907354 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:09:49.907316 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:09:50.034975 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:09:50.034939 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:09:56.893485 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:09:56.893439 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:09:57.002711 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:09:57.002675 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:09:57.082276 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:09:57.082243 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:09:57.612382 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:09:57.612342 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:09:57.685563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:09:57.685528 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:01.940602 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:10:01.940563 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:02.051752 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:10:02.051692 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:08.260796 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:10:08.260756 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:08.369224 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:10:08.369188 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:08.479343 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:10:08.479278 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:13.593137 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:10:13.593109 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:13.692569 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:10:13.692530 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:13.755759 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:10:13.755723 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:11:07.261175 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:07.261140 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:11:07.362413 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:07.362378 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:11:29.798439 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:29.798401 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:11:30.040840 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.040804 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw"] Sep 09 21:11:30.041056 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.041029 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fabe4da8-a1c9-4e4b-aff7-65756b9de477" containerName="registry" Sep 09 21:11:30.041056 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.041055 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fabe4da8-a1c9-4e4b-aff7-65756b9de477" containerName="registry" Sep 09 21:11:30.041134 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.041101 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fabe4da8-a1c9-4e4b-aff7-65756b9de477" containerName="registry" Sep 09 21:11:30.042850 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.042834 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" Sep 09 21:11:30.045640 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.045618 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 09 21:11:30.046649 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.046626 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zc2h9\"" Sep 09 21:11:30.046649 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.046642 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 09 21:11:30.055622 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.055595 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw"] Sep 09 21:11:30.135698 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.135658 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw\" (UID: \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" Sep 09 21:11:30.135891 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.135725 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-splhc\" (UniqueName: \"kubernetes.io/projected/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-kube-api-access-splhc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw\" (UID: \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" Sep 09 21:11:30.135891 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.135782 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw\" (UID: \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" Sep 09 21:11:30.236531 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.236491 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw\" (UID: \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" Sep 09 21:11:30.236531 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.236534 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw\" (UID: \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" Sep 09 21:11:30.236732 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.236582 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-splhc\" (UniqueName: \"kubernetes.io/projected/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-kube-api-access-splhc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw\" (UID: \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" Sep 09 21:11:30.236874 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.236854 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw\" (UID: \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" Sep 09 21:11:30.236912 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.236895 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw\" (UID: \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" Sep 09 21:11:30.246294 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.246269 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-splhc\" (UniqueName: \"kubernetes.io/projected/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-kube-api-access-splhc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw\" (UID: \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" Sep 09 21:11:30.352220 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.352119 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" Sep 09 21:11:30.635611 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.635531 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw"] Sep 09 21:11:30.638946 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:11:30.638921 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ad5d63_581d_41e3_9c0b_7937eaff2bb1.slice/crio-60d18b9dd5fd94f4b28648fb9cea46bfa6d84e1847efd5b8e3f26486ed8729e7 WatchSource:0}: Error finding container 60d18b9dd5fd94f4b28648fb9cea46bfa6d84e1847efd5b8e3f26486ed8729e7: Status 404 returned error can't find the container with id 60d18b9dd5fd94f4b28648fb9cea46bfa6d84e1847efd5b8e3f26486ed8729e7 Sep 09 21:11:30.641030 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.641010 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:11:30.903470 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:30.903376 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" event={"ID":"13ad5d63-581d-41e3-9c0b-7937eaff2bb1","Type":"ContainerStarted","Data":"60d18b9dd5fd94f4b28648fb9cea46bfa6d84e1847efd5b8e3f26486ed8729e7"} Sep 09 21:11:37.930382 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:37.930344 2560 generic.go:356] "Generic (PLEG): container finished" podID="13ad5d63-581d-41e3-9c0b-7937eaff2bb1" containerID="3d8416eefac11a6d686522838c646fc408528d8108cc577805011943ba820df4" exitCode=0 Sep 09 21:11:37.930889 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:37.930398 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" event={"ID":"13ad5d63-581d-41e3-9c0b-7937eaff2bb1","Type":"ContainerDied","Data":"3d8416eefac11a6d686522838c646fc408528d8108cc577805011943ba820df4"} Sep 09 21:11:55.994257 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:55.994225 2560 generic.go:356] "Generic (PLEG): container finished" podID="13ad5d63-581d-41e3-9c0b-7937eaff2bb1" containerID="7413a4d833ad66164cb7518e6cb6c03d8805e11e21ed4379625e819f8eda11ce" exitCode=0 Sep 09 21:11:55.994662 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:11:55.994269 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" event={"ID":"13ad5d63-581d-41e3-9c0b-7937eaff2bb1","Type":"ContainerDied","Data":"7413a4d833ad66164cb7518e6cb6c03d8805e11e21ed4379625e819f8eda11ce"} Sep 09 21:12:05.029124 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:05.029083 2560 generic.go:356] "Generic (PLEG): container finished" podID="13ad5d63-581d-41e3-9c0b-7937eaff2bb1" containerID="6c33bd940bf3eaa559c19a6f5ad46922f8cb31cca9b355994b37da6f18c7cfce" exitCode=0 Sep 09 21:12:05.029494 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:05.029160 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" event={"ID":"13ad5d63-581d-41e3-9c0b-7937eaff2bb1","Type":"ContainerDied","Data":"6c33bd940bf3eaa559c19a6f5ad46922f8cb31cca9b355994b37da6f18c7cfce"} Sep 09 21:12:06.153431 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:06.153405 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" Sep 09 21:12:06.328279 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:06.328185 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-splhc\" (UniqueName: \"kubernetes.io/projected/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-kube-api-access-splhc\") pod \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\" (UID: \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\") " Sep 09 21:12:06.328279 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:06.328232 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-bundle\") pod \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\" (UID: \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\") " Sep 09 21:12:06.328279 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:06.328275 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-util\") pod \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\" (UID: \"13ad5d63-581d-41e3-9c0b-7937eaff2bb1\") " Sep 09 21:12:06.328978 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:06.328949 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-bundle" (OuterVolumeSpecName: "bundle") pod "13ad5d63-581d-41e3-9c0b-7937eaff2bb1" (UID: "13ad5d63-581d-41e3-9c0b-7937eaff2bb1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:12:06.330546 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:06.330518 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-kube-api-access-splhc" (OuterVolumeSpecName: "kube-api-access-splhc") pod "13ad5d63-581d-41e3-9c0b-7937eaff2bb1" (UID: "13ad5d63-581d-41e3-9c0b-7937eaff2bb1"). InnerVolumeSpecName "kube-api-access-splhc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:12:06.332995 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:06.332975 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-util" (OuterVolumeSpecName: "util") pod "13ad5d63-581d-41e3-9c0b-7937eaff2bb1" (UID: "13ad5d63-581d-41e3-9c0b-7937eaff2bb1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:12:06.429548 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:06.429494 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-splhc\" (UniqueName: \"kubernetes.io/projected/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-kube-api-access-splhc\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:12:06.429548 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:06.429542 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-bundle\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:12:06.429548 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:06.429551 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13ad5d63-581d-41e3-9c0b-7937eaff2bb1-util\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:12:07.037615 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:07.037541 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" event={"ID":"13ad5d63-581d-41e3-9c0b-7937eaff2bb1","Type":"ContainerDied","Data":"60d18b9dd5fd94f4b28648fb9cea46bfa6d84e1847efd5b8e3f26486ed8729e7"} Sep 09 21:12:07.037615 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:07.037592 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60d18b9dd5fd94f4b28648fb9cea46bfa6d84e1847efd5b8e3f26486ed8729e7" Sep 09 21:12:07.037615 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:07.037559 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k5rmw" Sep 09 21:12:08.161319 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:08.161287 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:09.163646 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:09.163610 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:10.211145 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.211114 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2th5d"] Sep 09 21:12:10.211504 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.211332 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ad5d63-581d-41e3-9c0b-7937eaff2bb1" containerName="util" Sep 09 21:12:10.211504 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.211342 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ad5d63-581d-41e3-9c0b-7937eaff2bb1" containerName="util" Sep 09 21:12:10.211504 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.211360 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ad5d63-581d-41e3-9c0b-7937eaff2bb1" containerName="pull" Sep 09 21:12:10.211504 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.211365 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ad5d63-581d-41e3-9c0b-7937eaff2bb1" containerName="pull" Sep 09 21:12:10.211504 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.211371 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ad5d63-581d-41e3-9c0b-7937eaff2bb1" containerName="extract" Sep 09 21:12:10.211504 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.211377 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ad5d63-581d-41e3-9c0b-7937eaff2bb1" containerName="extract" Sep 09 21:12:10.211504 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.211415 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ad5d63-581d-41e3-9c0b-7937eaff2bb1" containerName="extract" Sep 09 21:12:10.213237 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.213221 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2th5d" Sep 09 21:12:10.216034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.216011 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Sep 09 21:12:10.216034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.216016 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:12:10.216227 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.216070 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-vjq8d\"" Sep 09 21:12:10.232098 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.232072 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2th5d"] Sep 09 21:12:10.355384 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.355342 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6sv7\" (UniqueName: \"kubernetes.io/projected/c2d154ca-c61b-4684-90c9-4d9fe31fb6b3-kube-api-access-z6sv7\") pod \"cert-manager-operator-controller-manager-55c444bf86-2th5d\" (UID: \"c2d154ca-c61b-4684-90c9-4d9fe31fb6b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2th5d" Sep 09 21:12:10.355573 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.355395 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2d154ca-c61b-4684-90c9-4d9fe31fb6b3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2th5d\" (UID: \"c2d154ca-c61b-4684-90c9-4d9fe31fb6b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2th5d" Sep 09 21:12:10.455916 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.455883 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6sv7\" (UniqueName: \"kubernetes.io/projected/c2d154ca-c61b-4684-90c9-4d9fe31fb6b3-kube-api-access-z6sv7\") pod \"cert-manager-operator-controller-manager-55c444bf86-2th5d\" (UID: \"c2d154ca-c61b-4684-90c9-4d9fe31fb6b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2th5d" Sep 09 21:12:10.456119 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.455938 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2d154ca-c61b-4684-90c9-4d9fe31fb6b3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2th5d\" (UID: \"c2d154ca-c61b-4684-90c9-4d9fe31fb6b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2th5d" Sep 09 21:12:10.456324 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.456305 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2d154ca-c61b-4684-90c9-4d9fe31fb6b3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2th5d\" (UID: \"c2d154ca-c61b-4684-90c9-4d9fe31fb6b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2th5d" Sep 09 21:12:10.465840 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.465794 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6sv7\" (UniqueName: \"kubernetes.io/projected/c2d154ca-c61b-4684-90c9-4d9fe31fb6b3-kube-api-access-z6sv7\") pod \"cert-manager-operator-controller-manager-55c444bf86-2th5d\" (UID: \"c2d154ca-c61b-4684-90c9-4d9fe31fb6b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2th5d" Sep 09 21:12:10.521630 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.521592 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2th5d" Sep 09 21:12:10.808095 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:10.808009 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2th5d"] Sep 09 21:12:10.811779 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:12:10.811750 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2d154ca_c61b_4684_90c9_4d9fe31fb6b3.slice/crio-fe504740a2f17b65574d7758a8f802e761be38fe2dffd7fc8522904e972bfc2a WatchSource:0}: Error finding container fe504740a2f17b65574d7758a8f802e761be38fe2dffd7fc8522904e972bfc2a: Status 404 returned error can't find the container with id fe504740a2f17b65574d7758a8f802e761be38fe2dffd7fc8522904e972bfc2a Sep 09 21:12:11.052013 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:11.051976 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2th5d" event={"ID":"c2d154ca-c61b-4684-90c9-4d9fe31fb6b3","Type":"ContainerStarted","Data":"fe504740a2f17b65574d7758a8f802e761be38fe2dffd7fc8522904e972bfc2a"} Sep 09 21:12:11.994642 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:11.994604 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:12.093625 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:12.093586 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:13.061647 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:13.061606 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2th5d" event={"ID":"c2d154ca-c61b-4684-90c9-4d9fe31fb6b3","Type":"ContainerStarted","Data":"98da8d5318c5cd116a6640c89b46856c56f9ce2a847b032e9e977db03350b2d6"} Sep 09 21:12:13.089533 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:13.089481 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2th5d" podStartSLOduration=1.043172867 podStartE2EDuration="3.089461776s" totalImagesPullingTime="2.046288909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:12:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:12:13.08672025 +0000 UTC m=+648.948242645" watchObservedRunningTime="2026-09-09 21:12:13.089461776 +0000 UTC m=+648.950984171" Sep 09 21:12:13.519284 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:13.519246 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:13.620848 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:13.620814 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:13.769618 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:13.769538 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:13.886239 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:13.886198 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:14.068642 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.068557 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl"] Sep 09 21:12:14.070634 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.070618 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" Sep 09 21:12:14.073258 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.073233 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 09 21:12:14.073258 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.073248 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zc2h9\"" Sep 09 21:12:14.074703 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.074685 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 09 21:12:14.088856 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.088835 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl"] Sep 09 21:12:14.186287 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.186249 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl\" (UID: \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" Sep 09 21:12:14.186511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.186357 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfhf4\" (UniqueName: \"kubernetes.io/projected/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-kube-api-access-zfhf4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl\" (UID: \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" Sep 09 21:12:14.186511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.186392 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl\" (UID: \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" Sep 09 21:12:14.287410 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.287371 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfhf4\" (UniqueName: \"kubernetes.io/projected/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-kube-api-access-zfhf4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl\" (UID: \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" Sep 09 21:12:14.287410 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.287417 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl\" (UID: \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" Sep 09 21:12:14.287618 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.287445 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl\" (UID: \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" Sep 09 21:12:14.287784 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.287769 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl\" (UID: \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" Sep 09 21:12:14.287846 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.287828 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl\" (UID: \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" Sep 09 21:12:14.297346 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.297320 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfhf4\" (UniqueName: \"kubernetes.io/projected/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-kube-api-access-zfhf4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl\" (UID: \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" Sep 09 21:12:14.380366 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.380275 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" Sep 09 21:12:14.654452 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:14.654370 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl"] Sep 09 21:12:14.657498 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:12:14.657473 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bba9bf7_55bf_4944_bd93_7e7b6e58531f.slice/crio-39400c859151e313486409599cf70eca2336a7a21b0996b594812a4a48ca409c WatchSource:0}: Error finding container 39400c859151e313486409599cf70eca2336a7a21b0996b594812a4a48ca409c: Status 404 returned error can't find the container with id 39400c859151e313486409599cf70eca2336a7a21b0996b594812a4a48ca409c Sep 09 21:12:15.070083 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:15.070024 2560 generic.go:356] "Generic (PLEG): container finished" podID="6bba9bf7-55bf-4944-bd93-7e7b6e58531f" containerID="28ae954c648a2a2689ae085a22126d887898e55ff2152ca7e3ceaadb354b5b7e" exitCode=0 Sep 09 21:12:15.070556 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:15.070113 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" event={"ID":"6bba9bf7-55bf-4944-bd93-7e7b6e58531f","Type":"ContainerDied","Data":"28ae954c648a2a2689ae085a22126d887898e55ff2152ca7e3ceaadb354b5b7e"} Sep 09 21:12:15.070556 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:15.070175 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" event={"ID":"6bba9bf7-55bf-4944-bd93-7e7b6e58531f","Type":"ContainerStarted","Data":"39400c859151e313486409599cf70eca2336a7a21b0996b594812a4a48ca409c"} Sep 09 21:12:15.947409 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:15.947350 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:16.364909 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:16.364872 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:17.080468 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:17.080434 2560 generic.go:356] "Generic (PLEG): container finished" podID="6bba9bf7-55bf-4944-bd93-7e7b6e58531f" containerID="aeed5d2a7356c22d57f5b6aaf9d6a2897b3f680d545d421db2d698efd6946724" exitCode=0 Sep 09 21:12:17.080607 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:17.080513 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" event={"ID":"6bba9bf7-55bf-4944-bd93-7e7b6e58531f","Type":"ContainerDied","Data":"aeed5d2a7356c22d57f5b6aaf9d6a2897b3f680d545d421db2d698efd6946724"} Sep 09 21:12:18.086437 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:18.086398 2560 generic.go:356] "Generic (PLEG): container finished" podID="6bba9bf7-55bf-4944-bd93-7e7b6e58531f" containerID="49e14f485ef5b5f2d2d6bed6d93f1abc607067b4a22915ecc1ea6d3708395e46" exitCode=0 Sep 09 21:12:18.086841 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:18.086488 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" event={"ID":"6bba9bf7-55bf-4944-bd93-7e7b6e58531f","Type":"ContainerDied","Data":"49e14f485ef5b5f2d2d6bed6d93f1abc607067b4a22915ecc1ea6d3708395e46"} Sep 09 21:12:19.210167 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:19.210145 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" Sep 09 21:12:19.325110 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:19.325059 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfhf4\" (UniqueName: \"kubernetes.io/projected/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-kube-api-access-zfhf4\") pod \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\" (UID: \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\") " Sep 09 21:12:19.325300 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:19.325128 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-util\") pod \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\" (UID: \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\") " Sep 09 21:12:19.325300 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:19.325180 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-bundle\") pod \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\" (UID: \"6bba9bf7-55bf-4944-bd93-7e7b6e58531f\") " Sep 09 21:12:19.325581 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:19.325559 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-bundle" (OuterVolumeSpecName: "bundle") pod "6bba9bf7-55bf-4944-bd93-7e7b6e58531f" (UID: "6bba9bf7-55bf-4944-bd93-7e7b6e58531f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:12:19.327166 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:19.327140 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-kube-api-access-zfhf4" (OuterVolumeSpecName: "kube-api-access-zfhf4") pod "6bba9bf7-55bf-4944-bd93-7e7b6e58531f" (UID: "6bba9bf7-55bf-4944-bd93-7e7b6e58531f"). InnerVolumeSpecName "kube-api-access-zfhf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:12:19.330803 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:19.330783 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-util" (OuterVolumeSpecName: "util") pod "6bba9bf7-55bf-4944-bd93-7e7b6e58531f" (UID: "6bba9bf7-55bf-4944-bd93-7e7b6e58531f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:12:19.426338 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:19.426249 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-bundle\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:12:19.426338 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:19.426278 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfhf4\" (UniqueName: \"kubernetes.io/projected/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-kube-api-access-zfhf4\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:12:19.426338 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:19.426288 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bba9bf7-55bf-4944-bd93-7e7b6e58531f-util\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:12:20.095164 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:20.095131 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" event={"ID":"6bba9bf7-55bf-4944-bd93-7e7b6e58531f","Type":"ContainerDied","Data":"39400c859151e313486409599cf70eca2336a7a21b0996b594812a4a48ca409c"} Sep 09 21:12:20.095380 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:20.095172 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2brsl" Sep 09 21:12:20.095380 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:20.095177 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39400c859151e313486409599cf70eca2336a7a21b0996b594812a4a48ca409c" Sep 09 21:12:21.802353 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:21.802316 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:21.896102 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:21.896070 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:22.008608 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.008567 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-27vmw"] Sep 09 21:12:22.008818 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.008806 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bba9bf7-55bf-4944-bd93-7e7b6e58531f" containerName="pull" Sep 09 21:12:22.008862 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.008821 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bba9bf7-55bf-4944-bd93-7e7b6e58531f" containerName="pull" Sep 09 21:12:22.008862 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.008832 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bba9bf7-55bf-4944-bd93-7e7b6e58531f" containerName="util" Sep 09 21:12:22.008862 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.008837 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bba9bf7-55bf-4944-bd93-7e7b6e58531f" containerName="util" Sep 09 21:12:22.008862 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.008843 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bba9bf7-55bf-4944-bd93-7e7b6e58531f" containerName="extract" Sep 09 21:12:22.008862 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.008849 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bba9bf7-55bf-4944-bd93-7e7b6e58531f" containerName="extract" Sep 09 21:12:22.009007 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.008892 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bba9bf7-55bf-4944-bd93-7e7b6e58531f" containerName="extract" Sep 09 21:12:22.011710 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.011693 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-27vmw" Sep 09 21:12:22.014432 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.014413 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-qrkml\"" Sep 09 21:12:22.014542 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.014463 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 09 21:12:22.015305 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.015287 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 09 21:12:22.024083 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.024063 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-27vmw"] Sep 09 21:12:22.145771 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.145668 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhr5t\" (UniqueName: \"kubernetes.io/projected/d2675414-576e-4172-939a-1cb5f6045f78-kube-api-access-jhr5t\") pod \"cert-manager-cainjector-559cdd8c87-27vmw\" (UID: \"d2675414-576e-4172-939a-1cb5f6045f78\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-27vmw" Sep 09 21:12:22.145771 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.145729 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2675414-576e-4172-939a-1cb5f6045f78-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-27vmw\" (UID: \"d2675414-576e-4172-939a-1cb5f6045f78\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-27vmw" Sep 09 21:12:22.246747 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.246711 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhr5t\" (UniqueName: \"kubernetes.io/projected/d2675414-576e-4172-939a-1cb5f6045f78-kube-api-access-jhr5t\") pod \"cert-manager-cainjector-559cdd8c87-27vmw\" (UID: \"d2675414-576e-4172-939a-1cb5f6045f78\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-27vmw" Sep 09 21:12:22.246919 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.246760 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2675414-576e-4172-939a-1cb5f6045f78-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-27vmw\" (UID: \"d2675414-576e-4172-939a-1cb5f6045f78\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-27vmw" Sep 09 21:12:22.256497 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.256471 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2675414-576e-4172-939a-1cb5f6045f78-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-27vmw\" (UID: \"d2675414-576e-4172-939a-1cb5f6045f78\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-27vmw" Sep 09 21:12:22.256645 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.256598 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhr5t\" (UniqueName: \"kubernetes.io/projected/d2675414-576e-4172-939a-1cb5f6045f78-kube-api-access-jhr5t\") pod \"cert-manager-cainjector-559cdd8c87-27vmw\" (UID: \"d2675414-576e-4172-939a-1cb5f6045f78\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-27vmw" Sep 09 21:12:22.320431 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.320393 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-27vmw" Sep 09 21:12:22.598861 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:22.598829 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-27vmw"] Sep 09 21:12:22.601862 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:12:22.601830 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2675414_576e_4172_939a_1cb5f6045f78.slice/crio-717529b38e4b25cc076f09bec2353b67d6f610743313b09a7dbb887b76496560 WatchSource:0}: Error finding container 717529b38e4b25cc076f09bec2353b67d6f610743313b09a7dbb887b76496560: Status 404 returned error can't find the container with id 717529b38e4b25cc076f09bec2353b67d6f610743313b09a7dbb887b76496560 Sep 09 21:12:23.107513 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:23.107470 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-27vmw" event={"ID":"d2675414-576e-4172-939a-1cb5f6045f78","Type":"ContainerStarted","Data":"717529b38e4b25cc076f09bec2353b67d6f610743313b09a7dbb887b76496560"} Sep 09 21:12:23.958523 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:23.958481 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:24.219031 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:24.218951 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4xhlr"] Sep 09 21:12:24.221909 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:24.221882 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4xhlr" Sep 09 21:12:24.226143 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:24.226120 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hdnfj\"" Sep 09 21:12:24.239482 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:24.239457 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4xhlr"] Sep 09 21:12:24.262446 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:24.262414 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc650100-6209-49a5-8cb8-9785eb691b34-bound-sa-token\") pod \"cert-manager-55b4d954d5-4xhlr\" (UID: \"cc650100-6209-49a5-8cb8-9785eb691b34\") " pod="cert-manager/cert-manager-55b4d954d5-4xhlr" Sep 09 21:12:24.262590 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:24.262471 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w29fs\" (UniqueName: \"kubernetes.io/projected/cc650100-6209-49a5-8cb8-9785eb691b34-kube-api-access-w29fs\") pod \"cert-manager-55b4d954d5-4xhlr\" (UID: \"cc650100-6209-49a5-8cb8-9785eb691b34\") " pod="cert-manager/cert-manager-55b4d954d5-4xhlr" Sep 09 21:12:24.363176 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:24.363138 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w29fs\" (UniqueName: \"kubernetes.io/projected/cc650100-6209-49a5-8cb8-9785eb691b34-kube-api-access-w29fs\") pod \"cert-manager-55b4d954d5-4xhlr\" (UID: \"cc650100-6209-49a5-8cb8-9785eb691b34\") " pod="cert-manager/cert-manager-55b4d954d5-4xhlr" Sep 09 21:12:24.363361 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:24.363232 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc650100-6209-49a5-8cb8-9785eb691b34-bound-sa-token\") pod \"cert-manager-55b4d954d5-4xhlr\" (UID: \"cc650100-6209-49a5-8cb8-9785eb691b34\") " pod="cert-manager/cert-manager-55b4d954d5-4xhlr" Sep 09 21:12:24.374355 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:24.374312 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc650100-6209-49a5-8cb8-9785eb691b34-bound-sa-token\") pod \"cert-manager-55b4d954d5-4xhlr\" (UID: \"cc650100-6209-49a5-8cb8-9785eb691b34\") " pod="cert-manager/cert-manager-55b4d954d5-4xhlr" Sep 09 21:12:24.374563 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:24.374542 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w29fs\" (UniqueName: \"kubernetes.io/projected/cc650100-6209-49a5-8cb8-9785eb691b34-kube-api-access-w29fs\") pod \"cert-manager-55b4d954d5-4xhlr\" (UID: \"cc650100-6209-49a5-8cb8-9785eb691b34\") " pod="cert-manager/cert-manager-55b4d954d5-4xhlr" Sep 09 21:12:24.536677 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:24.536588 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hdnfj\"" Sep 09 21:12:24.542670 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:24.542642 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4xhlr" Sep 09 21:12:24.831067 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:24.831019 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4xhlr"] Sep 09 21:12:25.584161 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:12:25.584124 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc650100_6209_49a5_8cb8_9785eb691b34.slice/crio-bcd014544c1a7c32add0c55cb65b357763cbd159ac3f672898abbf48fbf6e2ee WatchSource:0}: Error finding container bcd014544c1a7c32add0c55cb65b357763cbd159ac3f672898abbf48fbf6e2ee: Status 404 returned error can't find the container with id bcd014544c1a7c32add0c55cb65b357763cbd159ac3f672898abbf48fbf6e2ee Sep 09 21:12:26.121528 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:26.121494 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-27vmw" event={"ID":"d2675414-576e-4172-939a-1cb5f6045f78","Type":"ContainerStarted","Data":"0fca0d3dac36b11071a48132ada0b4e9cca7a7af72ce1ea13ba497f929b63d41"} Sep 09 21:12:26.123074 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:26.123052 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4xhlr" event={"ID":"cc650100-6209-49a5-8cb8-9785eb691b34","Type":"ContainerStarted","Data":"bcd014544c1a7c32add0c55cb65b357763cbd159ac3f672898abbf48fbf6e2ee"} Sep 09 21:12:26.142096 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:26.142018 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-27vmw" podStartSLOduration=2.099108746 podStartE2EDuration="5.141999848s" totalImagesPullingTime="3.042891102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:12:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:12:26.141638831 +0000 UTC m=+662.003161434" watchObservedRunningTime="2026-09-09 21:12:26.141999848 +0000 UTC m=+662.003522246" Sep 09 21:12:26.466520 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:26.466483 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:26.563181 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:26.563143 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:27.128747 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:27.128712 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4xhlr" event={"ID":"cc650100-6209-49a5-8cb8-9785eb691b34","Type":"ContainerStarted","Data":"e45f22b1d44726924b78c449c641d77892deeec7e3c68b0e4c871211b8e115ec"} Sep 09 21:12:27.147535 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:27.147478 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-4xhlr" podStartSLOduration=2.670888005 podStartE2EDuration="3.147459423s" totalImagesPullingTime="476.571418ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:12:27.147201791 +0000 UTC m=+663.008724189" watchObservedRunningTime="2026-09-09 21:12:27.147459423 +0000 UTC m=+663.008981820" Sep 09 21:12:27.368276 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:27.368237 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:27.464503 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:27.464467 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:29.249521 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.249487 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7"] Sep 09 21:12:29.252849 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.252829 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" Sep 09 21:12:29.255685 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.255668 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zc2h9\"" Sep 09 21:12:29.256004 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.255978 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 09 21:12:29.256140 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.256121 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 09 21:12:29.269152 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.269129 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7"] Sep 09 21:12:29.296407 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.296377 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a606704-66af-4d7a-be75-b906d02e1b46-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7\" (UID: \"3a606704-66af-4d7a-be75-b906d02e1b46\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" Sep 09 21:12:29.296514 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.296426 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a606704-66af-4d7a-be75-b906d02e1b46-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7\" (UID: \"3a606704-66af-4d7a-be75-b906d02e1b46\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" Sep 09 21:12:29.296514 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.296454 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj6th\" (UniqueName: \"kubernetes.io/projected/3a606704-66af-4d7a-be75-b906d02e1b46-kube-api-access-jj6th\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7\" (UID: \"3a606704-66af-4d7a-be75-b906d02e1b46\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" Sep 09 21:12:29.396956 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.396921 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a606704-66af-4d7a-be75-b906d02e1b46-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7\" (UID: \"3a606704-66af-4d7a-be75-b906d02e1b46\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" Sep 09 21:12:29.397133 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.396970 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a606704-66af-4d7a-be75-b906d02e1b46-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7\" (UID: \"3a606704-66af-4d7a-be75-b906d02e1b46\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" Sep 09 21:12:29.397133 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.396998 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj6th\" (UniqueName: \"kubernetes.io/projected/3a606704-66af-4d7a-be75-b906d02e1b46-kube-api-access-jj6th\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7\" (UID: \"3a606704-66af-4d7a-be75-b906d02e1b46\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" Sep 09 21:12:29.397323 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.397303 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a606704-66af-4d7a-be75-b906d02e1b46-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7\" (UID: \"3a606704-66af-4d7a-be75-b906d02e1b46\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" Sep 09 21:12:29.397361 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.397342 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a606704-66af-4d7a-be75-b906d02e1b46-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7\" (UID: \"3a606704-66af-4d7a-be75-b906d02e1b46\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" Sep 09 21:12:29.407781 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.407759 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj6th\" (UniqueName: \"kubernetes.io/projected/3a606704-66af-4d7a-be75-b906d02e1b46-kube-api-access-jj6th\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7\" (UID: \"3a606704-66af-4d7a-be75-b906d02e1b46\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" Sep 09 21:12:29.557334 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.557255 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:29.561101 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.561081 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" Sep 09 21:12:29.845384 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:29.845307 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7"] Sep 09 21:12:29.848177 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:12:29.848149 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a606704_66af_4d7a_be75_b906d02e1b46.slice/crio-b527f65c73c01f97dfec681b321c04b532dafb865c5fd2dd98a400655ed180c7 WatchSource:0}: Error finding container b527f65c73c01f97dfec681b321c04b532dafb865c5fd2dd98a400655ed180c7: Status 404 returned error can't find the container with id b527f65c73c01f97dfec681b321c04b532dafb865c5fd2dd98a400655ed180c7 Sep 09 21:12:30.141959 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:30.141877 2560 generic.go:356] "Generic (PLEG): container finished" podID="3a606704-66af-4d7a-be75-b906d02e1b46" containerID="7c06cb8047f5cda7c3abba96135c02f8c78a7370dee8c2dad534aa2a11f5aa7a" exitCode=0 Sep 09 21:12:30.141959 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:30.141921 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" event={"ID":"3a606704-66af-4d7a-be75-b906d02e1b46","Type":"ContainerDied","Data":"7c06cb8047f5cda7c3abba96135c02f8c78a7370dee8c2dad534aa2a11f5aa7a"} Sep 09 21:12:30.141959 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:30.141957 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" event={"ID":"3a606704-66af-4d7a-be75-b906d02e1b46","Type":"ContainerStarted","Data":"b527f65c73c01f97dfec681b321c04b532dafb865c5fd2dd98a400655ed180c7"} Sep 09 21:12:31.147682 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:31.147637 2560 generic.go:356] "Generic (PLEG): container finished" podID="3a606704-66af-4d7a-be75-b906d02e1b46" containerID="057b0c48ae5d755abce06e454cbab6f1ad7d9c6f6084b1c2479c8ff91cfeea75" exitCode=0 Sep 09 21:12:31.148127 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:31.147715 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" event={"ID":"3a606704-66af-4d7a-be75-b906d02e1b46","Type":"ContainerDied","Data":"057b0c48ae5d755abce06e454cbab6f1ad7d9c6f6084b1c2479c8ff91cfeea75"} Sep 09 21:12:32.153623 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:32.153590 2560 generic.go:356] "Generic (PLEG): container finished" podID="3a606704-66af-4d7a-be75-b906d02e1b46" containerID="f6ab8312d3abdb47bc1c2e10591fccd8f0bde51d230f8ea3a6ef12aff32ce29c" exitCode=0 Sep 09 21:12:32.154000 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:32.153629 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" event={"ID":"3a606704-66af-4d7a-be75-b906d02e1b46","Type":"ContainerDied","Data":"f6ab8312d3abdb47bc1c2e10591fccd8f0bde51d230f8ea3a6ef12aff32ce29c"} Sep 09 21:12:33.278753 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:33.278728 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" Sep 09 21:12:33.325183 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:33.325151 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a606704-66af-4d7a-be75-b906d02e1b46-util\") pod \"3a606704-66af-4d7a-be75-b906d02e1b46\" (UID: \"3a606704-66af-4d7a-be75-b906d02e1b46\") " Sep 09 21:12:33.325183 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:33.325183 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj6th\" (UniqueName: \"kubernetes.io/projected/3a606704-66af-4d7a-be75-b906d02e1b46-kube-api-access-jj6th\") pod \"3a606704-66af-4d7a-be75-b906d02e1b46\" (UID: \"3a606704-66af-4d7a-be75-b906d02e1b46\") " Sep 09 21:12:33.327186 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:33.327157 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a606704-66af-4d7a-be75-b906d02e1b46-kube-api-access-jj6th" (OuterVolumeSpecName: "kube-api-access-jj6th") pod "3a606704-66af-4d7a-be75-b906d02e1b46" (UID: "3a606704-66af-4d7a-be75-b906d02e1b46"). InnerVolumeSpecName "kube-api-access-jj6th". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:12:33.330009 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:33.329986 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a606704-66af-4d7a-be75-b906d02e1b46-util" (OuterVolumeSpecName: "util") pod "3a606704-66af-4d7a-be75-b906d02e1b46" (UID: "3a606704-66af-4d7a-be75-b906d02e1b46"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:12:33.425670 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:33.425585 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a606704-66af-4d7a-be75-b906d02e1b46-bundle\") pod \"3a606704-66af-4d7a-be75-b906d02e1b46\" (UID: \"3a606704-66af-4d7a-be75-b906d02e1b46\") " Sep 09 21:12:33.425815 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:33.425799 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a606704-66af-4d7a-be75-b906d02e1b46-util\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:12:33.425861 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:33.425821 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj6th\" (UniqueName: \"kubernetes.io/projected/3a606704-66af-4d7a-be75-b906d02e1b46-kube-api-access-jj6th\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:12:33.426339 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:33.426316 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a606704-66af-4d7a-be75-b906d02e1b46-bundle" (OuterVolumeSpecName: "bundle") pod "3a606704-66af-4d7a-be75-b906d02e1b46" (UID: "3a606704-66af-4d7a-be75-b906d02e1b46"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:12:33.526523 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:33.526489 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a606704-66af-4d7a-be75-b906d02e1b46-bundle\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:12:34.163686 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:34.163643 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" event={"ID":"3a606704-66af-4d7a-be75-b906d02e1b46","Type":"ContainerDied","Data":"b527f65c73c01f97dfec681b321c04b532dafb865c5fd2dd98a400655ed180c7"} Sep 09 21:12:34.163852 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:34.163700 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b527f65c73c01f97dfec681b321c04b532dafb865c5fd2dd98a400655ed180c7" Sep 09 21:12:34.163852 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:34.163654 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbrkx7" Sep 09 21:12:39.858747 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.858715 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv"] Sep 09 21:12:39.859233 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.858955 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a606704-66af-4d7a-be75-b906d02e1b46" containerName="util" Sep 09 21:12:39.859233 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.858966 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a606704-66af-4d7a-be75-b906d02e1b46" containerName="util" Sep 09 21:12:39.859233 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.858978 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a606704-66af-4d7a-be75-b906d02e1b46" containerName="extract" Sep 09 21:12:39.859233 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.858983 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a606704-66af-4d7a-be75-b906d02e1b46" containerName="extract" Sep 09 21:12:39.859233 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.858996 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a606704-66af-4d7a-be75-b906d02e1b46" containerName="pull" Sep 09 21:12:39.859233 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.859001 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a606704-66af-4d7a-be75-b906d02e1b46" containerName="pull" Sep 09 21:12:39.859233 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.859054 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a606704-66af-4d7a-be75-b906d02e1b46" containerName="extract" Sep 09 21:12:39.861810 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.861793 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" Sep 09 21:12:39.866629 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.866604 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zc2h9\"" Sep 09 21:12:39.866848 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.866829 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 09 21:12:39.867992 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.867703 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 09 21:12:39.870600 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.870581 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzvd2\" (UniqueName: \"kubernetes.io/projected/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-kube-api-access-wzvd2\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv\" (UID: \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" Sep 09 21:12:39.870662 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.870616 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv\" (UID: \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" Sep 09 21:12:39.870662 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.870648 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv\" (UID: \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" Sep 09 21:12:39.878971 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.878945 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv"] Sep 09 21:12:39.970996 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.970958 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzvd2\" (UniqueName: \"kubernetes.io/projected/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-kube-api-access-wzvd2\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv\" (UID: \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" Sep 09 21:12:39.970996 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.970998 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv\" (UID: \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" Sep 09 21:12:39.971221 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.971021 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv\" (UID: \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" Sep 09 21:12:39.971451 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.971430 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv\" (UID: \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" Sep 09 21:12:39.971451 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.971444 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv\" (UID: \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" Sep 09 21:12:39.987257 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:39.987232 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzvd2\" (UniqueName: \"kubernetes.io/projected/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-kube-api-access-wzvd2\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv\" (UID: \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" Sep 09 21:12:40.170942 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:40.170862 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" Sep 09 21:12:40.445678 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:40.445631 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv"] Sep 09 21:12:40.447674 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:12:40.447642 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2cf6d82_51cb_489b_85f1_6f15d0e3ac8b.slice/crio-8e595dd6dfe88c10667b793ce4866cb5b48ab0f8c64706ac08bda9fa0fb38acb WatchSource:0}: Error finding container 8e595dd6dfe88c10667b793ce4866cb5b48ab0f8c64706ac08bda9fa0fb38acb: Status 404 returned error can't find the container with id 8e595dd6dfe88c10667b793ce4866cb5b48ab0f8c64706ac08bda9fa0fb38acb Sep 09 21:12:40.922884 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:40.922849 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:41.012418 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:41.012387 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:41.194731 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:41.194640 2560 generic.go:356] "Generic (PLEG): container finished" podID="d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b" containerID="424ad5f0912e3e61615128bf959165d8d1f98a9807beba6ba8aa4b295aeef719" exitCode=0 Sep 09 21:12:41.194731 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:41.194680 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" event={"ID":"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b","Type":"ContainerDied","Data":"424ad5f0912e3e61615128bf959165d8d1f98a9807beba6ba8aa4b295aeef719"} Sep 09 21:12:41.194947 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:41.194735 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" event={"ID":"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b","Type":"ContainerStarted","Data":"8e595dd6dfe88c10667b793ce4866cb5b48ab0f8c64706ac08bda9fa0fb38acb"} Sep 09 21:12:41.568695 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:41.568662 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:41.861446 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:41.861410 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:41.967355 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:41.967323 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:41.988262 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:41.988231 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr"] Sep 09 21:12:41.990374 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:41.990356 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:41.992877 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:41.992854 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 09 21:12:41.993014 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:41.992902 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 09 21:12:41.993014 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:41.992968 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-c4wl6\"" Sep 09 21:12:41.993271 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:41.993254 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 09 21:12:41.993972 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:41.993957 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 09 21:12:42.006986 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.006963 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr"] Sep 09 21:12:42.089222 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.089128 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtkr5\" (UniqueName: \"kubernetes.io/projected/d9c661f8-fbaf-40a3-82c1-401cfa47885b-kube-api-access-wtkr5\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.089222 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.089172 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9c661f8-fbaf-40a3-82c1-401cfa47885b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.089222 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.089194 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9c661f8-fbaf-40a3-82c1-401cfa47885b-webhook-cert\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.089222 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.089212 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9c661f8-fbaf-40a3-82c1-401cfa47885b-tmp\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.089465 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.089244 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d9c661f8-fbaf-40a3-82c1-401cfa47885b-manifests\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.162575 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.162544 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:42.190606 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.190573 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtkr5\" (UniqueName: \"kubernetes.io/projected/d9c661f8-fbaf-40a3-82c1-401cfa47885b-kube-api-access-wtkr5\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.190755 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.190610 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9c661f8-fbaf-40a3-82c1-401cfa47885b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.190755 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.190631 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9c661f8-fbaf-40a3-82c1-401cfa47885b-webhook-cert\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.190755 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.190649 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9c661f8-fbaf-40a3-82c1-401cfa47885b-tmp\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.190929 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.190780 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d9c661f8-fbaf-40a3-82c1-401cfa47885b-manifests\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.191175 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.191147 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d9c661f8-fbaf-40a3-82c1-401cfa47885b-manifests\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.192990 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.192958 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9c661f8-fbaf-40a3-82c1-401cfa47885b-tmp\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.193135 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.193115 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9c661f8-fbaf-40a3-82c1-401cfa47885b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.193135 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.193123 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9c661f8-fbaf-40a3-82c1-401cfa47885b-webhook-cert\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.200197 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.200175 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtkr5\" (UniqueName: \"kubernetes.io/projected/d9c661f8-fbaf-40a3-82c1-401cfa47885b-kube-api-access-wtkr5\") pod \"opendatahub-operator-controller-manager-59685ff6b-jctfr\" (UID: \"d9c661f8-fbaf-40a3-82c1-401cfa47885b\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.201108 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.201083 2560 generic.go:356] "Generic (PLEG): container finished" podID="d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b" containerID="86b945dab53f0e5e15be71e0f8f2d14c2877fb68b7ea5fcfddd98e3d374556e9" exitCode=0 Sep 09 21:12:42.201192 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.201139 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" event={"ID":"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b","Type":"ContainerDied","Data":"86b945dab53f0e5e15be71e0f8f2d14c2877fb68b7ea5fcfddd98e3d374556e9"} Sep 09 21:12:42.260626 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.260596 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:42.299717 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.299682 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:42.361468 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.361262 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:42.460406 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.460376 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:42.570393 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.570362 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:42.594382 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.594350 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr"] Sep 09 21:12:42.597824 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:12:42.597802 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9c661f8_fbaf_40a3_82c1_401cfa47885b.slice/crio-a91ab180de1c5ba7add7bbac79fb3e9c20b712506743f9f9737f3075b7e27d61 WatchSource:0}: Error finding container a91ab180de1c5ba7add7bbac79fb3e9c20b712506743f9f9737f3075b7e27d61: Status 404 returned error can't find the container with id a91ab180de1c5ba7add7bbac79fb3e9c20b712506743f9f9737f3075b7e27d61 Sep 09 21:12:42.663485 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:42.663443 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:12:43.208553 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:43.208519 2560 generic.go:356] "Generic (PLEG): container finished" podID="d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b" containerID="7173387937fc579e89058ac5dd71cbe4f3e8c1bd943362d06485a774e9ca3698" exitCode=0 Sep 09 21:12:43.208950 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:43.208817 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" event={"ID":"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b","Type":"ContainerDied","Data":"7173387937fc579e89058ac5dd71cbe4f3e8c1bd943362d06485a774e9ca3698"} Sep 09 21:12:43.210101 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:43.210073 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" event={"ID":"d9c661f8-fbaf-40a3-82c1-401cfa47885b","Type":"ContainerStarted","Data":"a91ab180de1c5ba7add7bbac79fb3e9c20b712506743f9f9737f3075b7e27d61"} Sep 09 21:12:44.826744 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:44.826715 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" Sep 09 21:12:44.910392 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:44.910355 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-bundle\") pod \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\" (UID: \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\") " Sep 09 21:12:44.910556 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:44.910464 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzvd2\" (UniqueName: \"kubernetes.io/projected/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-kube-api-access-wzvd2\") pod \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\" (UID: \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\") " Sep 09 21:12:44.910556 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:44.910490 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-util\") pod \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\" (UID: \"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b\") " Sep 09 21:12:44.911271 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:44.911241 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-bundle" (OuterVolumeSpecName: "bundle") pod "d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b" (UID: "d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:12:44.912976 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:44.912952 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-kube-api-access-wzvd2" (OuterVolumeSpecName: "kube-api-access-wzvd2") pod "d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b" (UID: "d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b"). InnerVolumeSpecName "kube-api-access-wzvd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:12:44.916716 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:44.916688 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-util" (OuterVolumeSpecName: "util") pod "d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b" (UID: "d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:12:45.011587 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:45.011550 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzvd2\" (UniqueName: \"kubernetes.io/projected/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-kube-api-access-wzvd2\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:12:45.011587 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:45.011583 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-util\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:12:45.011774 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:45.011596 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b-bundle\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:12:45.221437 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:45.221404 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" Sep 09 21:12:45.221437 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:45.221419 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04hdhv" event={"ID":"d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b","Type":"ContainerDied","Data":"8e595dd6dfe88c10667b793ce4866cb5b48ab0f8c64706ac08bda9fa0fb38acb"} Sep 09 21:12:45.221668 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:45.221465 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e595dd6dfe88c10667b793ce4866cb5b48ab0f8c64706ac08bda9fa0fb38acb" Sep 09 21:12:46.226796 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:46.226760 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" event={"ID":"d9c661f8-fbaf-40a3-82c1-401cfa47885b","Type":"ContainerStarted","Data":"443ed7c60948dcaeb690f1cd2f974ceb9cbb86edfe78ec59373d0813aafefe1a"} Sep 09 21:12:47.233157 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:47.233118 2560 generic.go:356] "Generic (PLEG): container finished" podID="d9c661f8-fbaf-40a3-82c1-401cfa47885b" containerID="443ed7c60948dcaeb690f1cd2f974ceb9cbb86edfe78ec59373d0813aafefe1a" exitCode=0 Sep 09 21:12:47.233632 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:47.233176 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" event={"ID":"d9c661f8-fbaf-40a3-82c1-401cfa47885b","Type":"ContainerDied","Data":"443ed7c60948dcaeb690f1cd2f974ceb9cbb86edfe78ec59373d0813aafefe1a"} Sep 09 21:12:48.240092 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:48.240056 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" event={"ID":"d9c661f8-fbaf-40a3-82c1-401cfa47885b","Type":"ContainerStarted","Data":"ae190e6e98055de059d9955d28c0493527e1a4cfa164ae70028762d0e0d2c67e"} Sep 09 21:12:48.240670 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:48.240190 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:12:48.268889 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:48.268803 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" podStartSLOduration=2.655234627 podStartE2EDuration="7.268779607s" totalImagesPullingTime="3.558447907s" totalInitContainerRuntime="1.055097073s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:12:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:12:48.265236572 +0000 UTC m=+684.126758989" watchObservedRunningTime="2026-09-09 21:12:48.268779607 +0000 UTC m=+684.130302004" Sep 09 21:12:59.252535 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:12:59.252497 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-jctfr" Sep 09 21:13:01.650831 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:01.650792 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:01.758080 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:01.758015 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:01.951103 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:01.951069 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:02.018485 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.018444 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv"] Sep 09 21:13:02.018810 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.018786 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b" containerName="util" Sep 09 21:13:02.018810 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.018811 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b" containerName="util" Sep 09 21:13:02.018991 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.018823 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b" containerName="pull" Sep 09 21:13:02.018991 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.018831 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b" containerName="pull" Sep 09 21:13:02.018991 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.018859 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b" containerName="extract" Sep 09 21:13:02.018991 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.018868 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b" containerName="extract" Sep 09 21:13:02.018991 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.018932 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2cf6d82-51cb-489b-85f1-6f15d0e3ac8b" containerName="extract" Sep 09 21:13:02.025667 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.025632 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" Sep 09 21:13:02.028404 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.028374 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 09 21:13:02.028553 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.028537 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hqxr9\"" Sep 09 21:13:02.029621 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.029591 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 09 21:13:02.029728 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.029622 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 09 21:13:02.029728 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.029595 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 09 21:13:02.037877 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.037852 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv"] Sep 09 21:13:02.148502 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.148464 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84a4f911-9970-4e19-be48-6c6beff033a8-tmp\") pod \"kube-auth-proxy-55f7b6bff6-6mchv\" (UID: \"84a4f911-9970-4e19-be48-6c6beff033a8\") " pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" Sep 09 21:13:02.148670 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.148518 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84a4f911-9970-4e19-be48-6c6beff033a8-tls-certs\") pod \"kube-auth-proxy-55f7b6bff6-6mchv\" (UID: \"84a4f911-9970-4e19-be48-6c6beff033a8\") " pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" Sep 09 21:13:02.148670 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.148608 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8844\" (UniqueName: \"kubernetes.io/projected/84a4f911-9970-4e19-be48-6c6beff033a8-kube-api-access-r8844\") pod \"kube-auth-proxy-55f7b6bff6-6mchv\" (UID: \"84a4f911-9970-4e19-be48-6c6beff033a8\") " pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" Sep 09 21:13:02.249324 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.249235 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8844\" (UniqueName: \"kubernetes.io/projected/84a4f911-9970-4e19-be48-6c6beff033a8-kube-api-access-r8844\") pod \"kube-auth-proxy-55f7b6bff6-6mchv\" (UID: \"84a4f911-9970-4e19-be48-6c6beff033a8\") " pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" Sep 09 21:13:02.249324 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.249290 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84a4f911-9970-4e19-be48-6c6beff033a8-tmp\") pod \"kube-auth-proxy-55f7b6bff6-6mchv\" (UID: \"84a4f911-9970-4e19-be48-6c6beff033a8\") " pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" Sep 09 21:13:02.249324 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.249320 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84a4f911-9970-4e19-be48-6c6beff033a8-tls-certs\") pod \"kube-auth-proxy-55f7b6bff6-6mchv\" (UID: \"84a4f911-9970-4e19-be48-6c6beff033a8\") " pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" Sep 09 21:13:02.249555 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:13:02.249414 2560 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Sep 09 21:13:02.249555 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:13:02.249477 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84a4f911-9970-4e19-be48-6c6beff033a8-tls-certs podName:84a4f911-9970-4e19-be48-6c6beff033a8 nodeName:}" failed. No retries permitted until 2026-09-09 21:13:02.749459577 +0000 UTC m=+698.610981967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/84a4f911-9970-4e19-be48-6c6beff033a8-tls-certs") pod "kube-auth-proxy-55f7b6bff6-6mchv" (UID: "84a4f911-9970-4e19-be48-6c6beff033a8") : secret "kube-auth-proxy-tls" not found Sep 09 21:13:02.251730 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.251699 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84a4f911-9970-4e19-be48-6c6beff033a8-tmp\") pod \"kube-auth-proxy-55f7b6bff6-6mchv\" (UID: \"84a4f911-9970-4e19-be48-6c6beff033a8\") " pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" Sep 09 21:13:02.265830 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.265796 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8844\" (UniqueName: \"kubernetes.io/projected/84a4f911-9970-4e19-be48-6c6beff033a8-kube-api-access-r8844\") pod \"kube-auth-proxy-55f7b6bff6-6mchv\" (UID: \"84a4f911-9970-4e19-be48-6c6beff033a8\") " pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" Sep 09 21:13:02.754880 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.754840 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84a4f911-9970-4e19-be48-6c6beff033a8-tls-certs\") pod \"kube-auth-proxy-55f7b6bff6-6mchv\" (UID: \"84a4f911-9970-4e19-be48-6c6beff033a8\") " pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" Sep 09 21:13:02.755384 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:13:02.755019 2560 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Sep 09 21:13:02.755384 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:13:02.755140 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84a4f911-9970-4e19-be48-6c6beff033a8-tls-certs podName:84a4f911-9970-4e19-be48-6c6beff033a8 nodeName:}" failed. No retries permitted until 2026-09-09 21:13:03.755117864 +0000 UTC m=+699.616640251 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/84a4f911-9970-4e19-be48-6c6beff033a8-tls-certs") pod "kube-auth-proxy-55f7b6bff6-6mchv" (UID: "84a4f911-9970-4e19-be48-6c6beff033a8") : secret "kube-auth-proxy-tls" not found Sep 09 21:13:02.997529 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:02.997494 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:03.674177 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.674143 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:03.685418 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.685381 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg"] Sep 09 21:13:03.691310 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.691286 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.694053 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.694022 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 09 21:13:03.694181 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.694063 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 09 21:13:03.694181 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.694019 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 09 21:13:03.694181 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.694116 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 09 21:13:03.694420 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.694405 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-m9nvg\"" Sep 09 21:13:03.708409 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.708386 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg"] Sep 09 21:13:03.764923 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.764887 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.765326 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.764929 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.765326 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.764958 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.765326 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.765028 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84a4f911-9970-4e19-be48-6c6beff033a8-tls-certs\") pod \"kube-auth-proxy-55f7b6bff6-6mchv\" (UID: \"84a4f911-9970-4e19-be48-6c6beff033a8\") " pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" Sep 09 21:13:03.765326 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.765077 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.765326 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.765096 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.765326 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.765116 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.765326 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.765138 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2pwc\" (UniqueName: \"kubernetes.io/projected/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-kube-api-access-c2pwc\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.767408 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.767389 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84a4f911-9970-4e19-be48-6c6beff033a8-tls-certs\") pod \"kube-auth-proxy-55f7b6bff6-6mchv\" (UID: \"84a4f911-9970-4e19-be48-6c6beff033a8\") " pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" Sep 09 21:13:03.788745 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.788710 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:03.835730 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.835692 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" Sep 09 21:13:03.866005 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.865962 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.866197 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.866022 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.866197 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.866093 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.866197 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.866135 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.866197 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.866159 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.866197 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.866189 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.866462 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.866220 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2pwc\" (UniqueName: \"kubernetes.io/projected/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-kube-api-access-c2pwc\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.866750 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.866712 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.868619 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.868463 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.868721 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.868649 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.868768 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.868720 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.868868 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.868852 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.875884 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.875857 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:03.880539 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:03.880505 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2pwc\" (UniqueName: \"kubernetes.io/projected/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-kube-api-access-c2pwc\") pod \"istiod-openshift-gateway-7b7995d5cd-jzbgg\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:04.001230 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:04.001187 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:04.030162 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:04.030125 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:04.126607 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:04.126575 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv"] Sep 09 21:13:04.129551 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:13:04.129521 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a4f911_9970_4e19_be48_6c6beff033a8.slice/crio-caead909cc2f6c72652b2baeec2fcef53fb846deb56bf765e8d6564ae3d823f4 WatchSource:0}: Error finding container caead909cc2f6c72652b2baeec2fcef53fb846deb56bf765e8d6564ae3d823f4: Status 404 returned error can't find the container with id caead909cc2f6c72652b2baeec2fcef53fb846deb56bf765e8d6564ae3d823f4 Sep 09 21:13:04.168764 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:04.168725 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:04.284585 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:04.284531 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg"] Sep 09 21:13:04.287196 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:13:04.287166 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f4b21d2_8b9b_4d43_8a5a_2d52b3821e18.slice/crio-0be60916330d052912895089db2d315d036d06d326ccf5cb9be575408b3cbac8 WatchSource:0}: Error finding container 0be60916330d052912895089db2d315d036d06d326ccf5cb9be575408b3cbac8: Status 404 returned error can't find the container with id 0be60916330d052912895089db2d315d036d06d326ccf5cb9be575408b3cbac8 Sep 09 21:13:04.311987 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:04.311955 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" event={"ID":"84a4f911-9970-4e19-be48-6c6beff033a8","Type":"ContainerStarted","Data":"caead909cc2f6c72652b2baeec2fcef53fb846deb56bf765e8d6564ae3d823f4"} Sep 09 21:13:04.313080 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:04.313035 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" event={"ID":"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18","Type":"ContainerStarted","Data":"0be60916330d052912895089db2d315d036d06d326ccf5cb9be575408b3cbac8"} Sep 09 21:13:04.999229 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:04.999183 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:05.401932 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:05.401847 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:07.330895 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:07.330862 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" event={"ID":"84a4f911-9970-4e19-be48-6c6beff033a8","Type":"ContainerStarted","Data":"abffa8e09e11af75869f3fa835ce668da76f2e81f2c900feb5afe4687e2c00c0"} Sep 09 21:13:07.353110 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:07.353058 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-55f7b6bff6-6mchv" podStartSLOduration=3.721624952 podStartE2EDuration="6.353026242s" totalImagesPullingTime="2.63140129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:13:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:13:07.352443066 +0000 UTC m=+703.213965461" watchObservedRunningTime="2026-09-09 21:13:07.353026242 +0000 UTC m=+703.214548636" Sep 09 21:13:12.661685 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:12.661647 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:12.767078 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:12.767026 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:12.867413 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:12.867375 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:13.209203 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:13.209117 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:13.435030 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:13.434990 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:14.752961 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:14.752908 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 09 21:13:14.753236 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:14.753005 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 09 21:13:15.370235 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:15.370192 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" event={"ID":"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18","Type":"ContainerStarted","Data":"e04ac4f8bd309223d5c09bfcd61e6df85b4e575ef49f23df4ba4b5d28515150c"} Sep 09 21:13:15.370442 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:15.370367 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:15.372176 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:15.372151 2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-jzbgg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 09 21:13:15.372300 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:15.372197 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" podUID="4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:15.404216 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:15.404145 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" podStartSLOduration=1.9407126209999999 podStartE2EDuration="12.404123607s" totalImagesPullingTime="10.463410986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:13:15.398960574 +0000 UTC m=+711.260482998" watchObservedRunningTime="2026-09-09 21:13:15.404123607 +0000 UTC m=+711.265646003" Sep 09 21:13:15.718907 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:15.718841 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:15.822847 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:15.822815 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:16.375593 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:16.375562 2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-jzbgg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 09 21:13:16.375756 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:16.375610 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" podUID="4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:16.900677 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:16.900575 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:17.050932 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:17.050889 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:17.222684 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:17.222649 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:19.376168 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:19.376135 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:25.009554 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.009516 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:25.139372 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.139337 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:25.175552 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.175519 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8bh26"] Sep 09 21:13:25.179080 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.179036 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8bh26" Sep 09 21:13:25.182308 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.182285 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mg64x\"" Sep 09 21:13:25.183284 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.183258 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z"] Sep 09 21:13:25.184106 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.183629 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 09 21:13:25.185726 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.185708 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z" Sep 09 21:13:25.193984 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.193959 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8bh26"] Sep 09 21:13:25.196820 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.196549 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wcqnf\"" Sep 09 21:13:25.196820 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.196562 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Sep 09 21:13:25.211810 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.211601 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z"] Sep 09 21:13:25.244912 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.244854 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:25.344687 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.344573 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d915fcaa-3a64-40d0-917f-b07ee74e8a53-cert\") pod \"kserve-controller-manager-d6f4b65b8-8bh26\" (UID: \"d915fcaa-3a64-40d0-917f-b07ee74e8a53\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8bh26" Sep 09 21:13:25.344875 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.344719 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn2nj\" (UniqueName: \"kubernetes.io/projected/d915fcaa-3a64-40d0-917f-b07ee74e8a53-kube-api-access-zn2nj\") pod \"kserve-controller-manager-d6f4b65b8-8bh26\" (UID: \"d915fcaa-3a64-40d0-917f-b07ee74e8a53\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8bh26" Sep 09 21:13:25.344875 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.344760 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgj9f\" (UniqueName: \"kubernetes.io/projected/e856b08f-2e82-4dd4-a929-ba32adcc282a-kube-api-access-cgj9f\") pod \"llmisvc-controller-manager-777c6bd7d5-mb75z\" (UID: \"e856b08f-2e82-4dd4-a929-ba32adcc282a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z" Sep 09 21:13:25.344875 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.344794 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e856b08f-2e82-4dd4-a929-ba32adcc282a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mb75z\" (UID: \"e856b08f-2e82-4dd4-a929-ba32adcc282a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z" Sep 09 21:13:25.446104 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.446070 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d915fcaa-3a64-40d0-917f-b07ee74e8a53-cert\") pod \"kserve-controller-manager-d6f4b65b8-8bh26\" (UID: \"d915fcaa-3a64-40d0-917f-b07ee74e8a53\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8bh26" Sep 09 21:13:25.446298 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.446140 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn2nj\" (UniqueName: \"kubernetes.io/projected/d915fcaa-3a64-40d0-917f-b07ee74e8a53-kube-api-access-zn2nj\") pod \"kserve-controller-manager-d6f4b65b8-8bh26\" (UID: \"d915fcaa-3a64-40d0-917f-b07ee74e8a53\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8bh26" Sep 09 21:13:25.446298 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.446168 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgj9f\" (UniqueName: \"kubernetes.io/projected/e856b08f-2e82-4dd4-a929-ba32adcc282a-kube-api-access-cgj9f\") pod \"llmisvc-controller-manager-777c6bd7d5-mb75z\" (UID: \"e856b08f-2e82-4dd4-a929-ba32adcc282a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z" Sep 09 21:13:25.446298 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.446195 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e856b08f-2e82-4dd4-a929-ba32adcc282a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mb75z\" (UID: \"e856b08f-2e82-4dd4-a929-ba32adcc282a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z" Sep 09 21:13:25.446298 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:13:25.446243 2560 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Sep 09 21:13:25.446568 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:13:25.446317 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d915fcaa-3a64-40d0-917f-b07ee74e8a53-cert podName:d915fcaa-3a64-40d0-917f-b07ee74e8a53 nodeName:}" failed. No retries permitted until 2026-09-09 21:13:25.946296707 +0000 UTC m=+721.807819084 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d915fcaa-3a64-40d0-917f-b07ee74e8a53-cert") pod "kserve-controller-manager-d6f4b65b8-8bh26" (UID: "d915fcaa-3a64-40d0-917f-b07ee74e8a53") : secret "kserve-webhook-server-cert" not found Sep 09 21:13:25.448917 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.448884 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e856b08f-2e82-4dd4-a929-ba32adcc282a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mb75z\" (UID: \"e856b08f-2e82-4dd4-a929-ba32adcc282a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z" Sep 09 21:13:25.488719 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.488688 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn2nj\" (UniqueName: \"kubernetes.io/projected/d915fcaa-3a64-40d0-917f-b07ee74e8a53-kube-api-access-zn2nj\") pod \"kserve-controller-manager-d6f4b65b8-8bh26\" (UID: \"d915fcaa-3a64-40d0-917f-b07ee74e8a53\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8bh26" Sep 09 21:13:25.514113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.514078 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgj9f\" (UniqueName: \"kubernetes.io/projected/e856b08f-2e82-4dd4-a929-ba32adcc282a-kube-api-access-cgj9f\") pod \"llmisvc-controller-manager-777c6bd7d5-mb75z\" (UID: \"e856b08f-2e82-4dd4-a929-ba32adcc282a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z" Sep 09 21:13:25.555586 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.555549 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:25.730548 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.730510 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:25.803514 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.803474 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z" Sep 09 21:13:25.904582 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.904542 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:25.950595 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.950560 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d915fcaa-3a64-40d0-917f-b07ee74e8a53-cert\") pod \"kserve-controller-manager-d6f4b65b8-8bh26\" (UID: \"d915fcaa-3a64-40d0-917f-b07ee74e8a53\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8bh26" Sep 09 21:13:25.952993 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:25.952974 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d915fcaa-3a64-40d0-917f-b07ee74e8a53-cert\") pod \"kserve-controller-manager-d6f4b65b8-8bh26\" (UID: \"d915fcaa-3a64-40d0-917f-b07ee74e8a53\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8bh26" Sep 09 21:13:26.090457 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:26.090431 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z"] Sep 09 21:13:26.091527 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:13:26.091504 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode856b08f_2e82_4dd4_a929_ba32adcc282a.slice/crio-7728868bf883a4949fa6514f76e0606a7eb0041fee79de0c9519382b6949f5a4 WatchSource:0}: Error finding container 7728868bf883a4949fa6514f76e0606a7eb0041fee79de0c9519382b6949f5a4: Status 404 returned error can't find the container with id 7728868bf883a4949fa6514f76e0606a7eb0041fee79de0c9519382b6949f5a4 Sep 09 21:13:26.098092 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:26.098032 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8bh26" Sep 09 21:13:26.226538 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:26.226503 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:26.375794 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:26.375769 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8bh26"] Sep 09 21:13:26.377539 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:13:26.377514 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd915fcaa_3a64_40d0_917f_b07ee74e8a53.slice/crio-1f6bad92ece9fb1b0c2e29deb72d97f1465bb23e096b24f7a8b721888f023659 WatchSource:0}: Error finding container 1f6bad92ece9fb1b0c2e29deb72d97f1465bb23e096b24f7a8b721888f023659: Status 404 returned error can't find the container with id 1f6bad92ece9fb1b0c2e29deb72d97f1465bb23e096b24f7a8b721888f023659 Sep 09 21:13:26.419628 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:26.419596 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z" event={"ID":"e856b08f-2e82-4dd4-a929-ba32adcc282a","Type":"ContainerStarted","Data":"7728868bf883a4949fa6514f76e0606a7eb0041fee79de0c9519382b6949f5a4"} Sep 09 21:13:26.420823 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:26.420794 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8bh26" event={"ID":"d915fcaa-3a64-40d0-917f-b07ee74e8a53","Type":"ContainerStarted","Data":"1f6bad92ece9fb1b0c2e29deb72d97f1465bb23e096b24f7a8b721888f023659"} Sep 09 21:13:28.167773 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:28.167709 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:28.542435 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:28.542392 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg"] Sep 09 21:13:28.542701 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:28.542656 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" podUID="4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" containerName="discovery" containerID="cri-o://e04ac4f8bd309223d5c09bfcd61e6df85b4e575ef49f23df4ba4b5d28515150c" gracePeriod=30 Sep 09 21:13:28.986149 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:28.986119 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:29.083306 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.083273 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-token\") pod \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " Sep 09 21:13:29.083487 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.083322 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-kubeconfig\") pod \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " Sep 09 21:13:29.083487 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.083369 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2pwc\" (UniqueName: \"kubernetes.io/projected/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-kube-api-access-c2pwc\") pod \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " Sep 09 21:13:29.083487 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.083401 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-local-certs\") pod \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " Sep 09 21:13:29.083633 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.083537 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-cacerts\") pod \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " Sep 09 21:13:29.083633 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.083592 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-csr-ca-configmap\") pod \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " Sep 09 21:13:29.083750 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.083654 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-csr-dns-cert\") pod \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\" (UID: \"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18\") " Sep 09 21:13:29.084060 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.084003 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" (UID: "4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:13:29.085947 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.085924 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" (UID: "4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:13:29.086089 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.085948 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" (UID: "4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:13:29.086164 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.086079 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-cacerts" (OuterVolumeSpecName: "cacerts") pod "4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" (UID: "4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:13:29.086164 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.086102 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-kube-api-access-c2pwc" (OuterVolumeSpecName: "kube-api-access-c2pwc") pod "4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" (UID: "4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18"). InnerVolumeSpecName "kube-api-access-c2pwc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:13:29.086164 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.086134 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-local-certs" (OuterVolumeSpecName: "local-certs") pod "4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" (UID: "4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:13:29.086164 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.086157 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-token" (OuterVolumeSpecName: "istio-token") pod "4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" (UID: "4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:13:29.184358 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.184317 2560 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-kubeconfig\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:13:29.184358 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.184350 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2pwc\" (UniqueName: \"kubernetes.io/projected/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-kube-api-access-c2pwc\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:13:29.184358 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.184364 2560 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-local-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:13:29.184818 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.184375 2560 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-cacerts\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:13:29.184818 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.184386 2560 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-csr-ca-configmap\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:13:29.184818 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.184397 2560 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-csr-dns-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:13:29.184818 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.184408 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18-istio-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:13:29.440212 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.440111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8bh26" event={"ID":"d915fcaa-3a64-40d0-917f-b07ee74e8a53","Type":"ContainerStarted","Data":"53d6fc130704e906705987da0d6ae70c3ff248d27926fb83f0499fc66c533d2f"} Sep 09 21:13:29.440380 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.440278 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8bh26" Sep 09 21:13:29.441618 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.441591 2560 generic.go:356] "Generic (PLEG): container finished" podID="4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" containerID="e04ac4f8bd309223d5c09bfcd61e6df85b4e575ef49f23df4ba4b5d28515150c" exitCode=0 Sep 09 21:13:29.441743 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.441639 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" Sep 09 21:13:29.441743 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.441648 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" event={"ID":"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18","Type":"ContainerDied","Data":"e04ac4f8bd309223d5c09bfcd61e6df85b4e575ef49f23df4ba4b5d28515150c"} Sep 09 21:13:29.441743 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.441691 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg" event={"ID":"4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18","Type":"ContainerDied","Data":"0be60916330d052912895089db2d315d036d06d326ccf5cb9be575408b3cbac8"} Sep 09 21:13:29.441743 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.441719 2560 scope.go:117] "RemoveContainer" containerID="e04ac4f8bd309223d5c09bfcd61e6df85b4e575ef49f23df4ba4b5d28515150c" Sep 09 21:13:29.466078 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.465730 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8bh26" podStartSLOduration=1.979990986 podStartE2EDuration="4.465709992s" totalImagesPullingTime="2.485719006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:13:29.46423034 +0000 UTC m=+725.325752736" watchObservedRunningTime="2026-09-09 21:13:29.465709992 +0000 UTC m=+725.327232390" Sep 09 21:13:29.501835 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.501796 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg"] Sep 09 21:13:29.505588 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.505557 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jzbgg"] Sep 09 21:13:29.797876 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.797693 2560 scope.go:117] "RemoveContainer" containerID="e04ac4f8bd309223d5c09bfcd61e6df85b4e575ef49f23df4ba4b5d28515150c" Sep 09 21:13:29.798144 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:13:29.798110 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e04ac4f8bd309223d5c09bfcd61e6df85b4e575ef49f23df4ba4b5d28515150c\": container with ID starting with e04ac4f8bd309223d5c09bfcd61e6df85b4e575ef49f23df4ba4b5d28515150c not found: ID does not exist" containerID="e04ac4f8bd309223d5c09bfcd61e6df85b4e575ef49f23df4ba4b5d28515150c" Sep 09 21:13:29.798252 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:29.798160 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e04ac4f8bd309223d5c09bfcd61e6df85b4e575ef49f23df4ba4b5d28515150c"} err="failed to get container status \"e04ac4f8bd309223d5c09bfcd61e6df85b4e575ef49f23df4ba4b5d28515150c\": rpc error: code = NotFound desc = could not find container \"e04ac4f8bd309223d5c09bfcd61e6df85b4e575ef49f23df4ba4b5d28515150c\": container with ID starting with e04ac4f8bd309223d5c09bfcd61e6df85b4e575ef49f23df4ba4b5d28515150c not found: ID does not exist" Sep 09 21:13:30.448785 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:30.448739 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z" event={"ID":"e856b08f-2e82-4dd4-a929-ba32adcc282a","Type":"ContainerStarted","Data":"c68ef3b2a0a535879aa158a1f0a5333050ce2111359d7e059a133d7d5632ffbb"} Sep 09 21:13:30.449350 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:30.448871 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z" Sep 09 21:13:30.481003 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:30.480911 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z" podStartSLOduration=1.730452486 podStartE2EDuration="5.480888536s" totalImagesPullingTime="3.75043605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:13:30.47924411 +0000 UTC m=+726.340766506" watchObservedRunningTime="2026-09-09 21:13:30.480888536 +0000 UTC m=+726.342410932" Sep 09 21:13:30.779842 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:30.779811 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" path="/var/lib/kubelet/pods/4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18/volumes" Sep 09 21:13:45.467842 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:45.467801 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:47.009740 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:47.009709 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:47.109675 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:47.109640 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:13:53.371182 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:13:53.371146 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:00.455019 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:00.454983 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8bh26" Sep 09 21:14:01.459082 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:01.459023 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mb75z" Sep 09 21:14:01.572897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:01.572865 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:01.657305 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:01.657270 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:12.233753 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:12.233711 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:12.339603 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:12.339575 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:12.445631 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:12.445588 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:13.799596 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:13.799555 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:13.882972 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:13.882934 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:18.461515 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.461478 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9"] Sep 09 21:14:18.461922 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.461750 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" containerName="discovery" Sep 09 21:14:18.461922 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.461761 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" containerName="discovery" Sep 09 21:14:18.461922 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.461837 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f4b21d2-8b9b-4d43-8a5a-2d52b3821e18" containerName="discovery" Sep 09 21:14:18.466649 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.466633 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" Sep 09 21:14:18.470283 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.470260 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 09 21:14:18.471203 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.471183 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zc2h9\"" Sep 09 21:14:18.471203 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.471196 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 09 21:14:18.477257 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.477235 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9"] Sep 09 21:14:18.551822 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.551783 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897"] Sep 09 21:14:18.554400 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.554380 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" Sep 09 21:14:18.568443 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.568419 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897"] Sep 09 21:14:18.573255 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.573231 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83e41dd2-a885-4441-9f5b-b9325a3aecaa-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9\" (UID: \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" Sep 09 21:14:18.573348 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.573285 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83e41dd2-a885-4441-9f5b-b9325a3aecaa-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9\" (UID: \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" Sep 09 21:14:18.573387 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.573368 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv8jg\" (UniqueName: \"kubernetes.io/projected/83e41dd2-a885-4441-9f5b-b9325a3aecaa-kube-api-access-xv8jg\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9\" (UID: \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" Sep 09 21:14:18.644685 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.644640 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf"] Sep 09 21:14:18.647191 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.647164 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" Sep 09 21:14:18.659455 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.659427 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf"] Sep 09 21:14:18.673895 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.673868 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83e41dd2-a885-4441-9f5b-b9325a3aecaa-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9\" (UID: \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" Sep 09 21:14:18.674076 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.673919 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83e41dd2-a885-4441-9f5b-b9325a3aecaa-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9\" (UID: \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" Sep 09 21:14:18.674076 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.673951 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l5vj\" (UniqueName: \"kubernetes.io/projected/29f5f7a1-026b-4121-980e-20de06302bcb-kube-api-access-5l5vj\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897\" (UID: \"29f5f7a1-026b-4121-980e-20de06302bcb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" Sep 09 21:14:18.674076 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.673976 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29f5f7a1-026b-4121-980e-20de06302bcb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897\" (UID: \"29f5f7a1-026b-4121-980e-20de06302bcb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" Sep 09 21:14:18.674076 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.673996 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv8jg\" (UniqueName: \"kubernetes.io/projected/83e41dd2-a885-4441-9f5b-b9325a3aecaa-kube-api-access-xv8jg\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9\" (UID: \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" Sep 09 21:14:18.674076 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.674012 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29f5f7a1-026b-4121-980e-20de06302bcb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897\" (UID: \"29f5f7a1-026b-4121-980e-20de06302bcb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" Sep 09 21:14:18.674283 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.674262 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83e41dd2-a885-4441-9f5b-b9325a3aecaa-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9\" (UID: \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" Sep 09 21:14:18.674360 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.674344 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83e41dd2-a885-4441-9f5b-b9325a3aecaa-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9\" (UID: \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" Sep 09 21:14:18.684707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.684679 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv8jg\" (UniqueName: \"kubernetes.io/projected/83e41dd2-a885-4441-9f5b-b9325a3aecaa-kube-api-access-xv8jg\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9\" (UID: \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" Sep 09 21:14:18.775036 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.774942 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" Sep 09 21:14:18.775221 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.775180 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l5vj\" (UniqueName: \"kubernetes.io/projected/29f5f7a1-026b-4121-980e-20de06302bcb-kube-api-access-5l5vj\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897\" (UID: \"29f5f7a1-026b-4121-980e-20de06302bcb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" Sep 09 21:14:18.775264 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.775232 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29f5f7a1-026b-4121-980e-20de06302bcb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897\" (UID: \"29f5f7a1-026b-4121-980e-20de06302bcb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" Sep 09 21:14:18.775299 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.775260 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29f5f7a1-026b-4121-980e-20de06302bcb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897\" (UID: \"29f5f7a1-026b-4121-980e-20de06302bcb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" Sep 09 21:14:18.775335 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.775312 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf\" (UID: \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" Sep 09 21:14:18.775429 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.775339 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf\" (UID: \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" Sep 09 21:14:18.775429 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.775364 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f66kh\" (UniqueName: \"kubernetes.io/projected/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-kube-api-access-f66kh\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf\" (UID: \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" Sep 09 21:14:18.775659 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.775640 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29f5f7a1-026b-4121-980e-20de06302bcb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897\" (UID: \"29f5f7a1-026b-4121-980e-20de06302bcb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" Sep 09 21:14:18.775701 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.775661 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29f5f7a1-026b-4121-980e-20de06302bcb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897\" (UID: \"29f5f7a1-026b-4121-980e-20de06302bcb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" Sep 09 21:14:18.786556 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.786522 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l5vj\" (UniqueName: \"kubernetes.io/projected/29f5f7a1-026b-4121-980e-20de06302bcb-kube-api-access-5l5vj\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897\" (UID: \"29f5f7a1-026b-4121-980e-20de06302bcb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" Sep 09 21:14:18.787133 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.787112 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr"] Sep 09 21:14:18.789754 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.789736 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" Sep 09 21:14:18.802447 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.802410 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr"] Sep 09 21:14:18.863361 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.863324 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" Sep 09 21:14:18.875964 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.875932 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c074b89-8236-430e-a2f4-70a3921a956b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr\" (UID: \"3c074b89-8236-430e-a2f4-70a3921a956b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" Sep 09 21:14:18.876080 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.875994 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c074b89-8236-430e-a2f4-70a3921a956b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr\" (UID: \"3c074b89-8236-430e-a2f4-70a3921a956b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" Sep 09 21:14:18.876080 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.876054 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgnmn\" (UniqueName: \"kubernetes.io/projected/3c074b89-8236-430e-a2f4-70a3921a956b-kube-api-access-kgnmn\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr\" (UID: \"3c074b89-8236-430e-a2f4-70a3921a956b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" Sep 09 21:14:18.876859 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.876834 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf\" (UID: \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" Sep 09 21:14:18.876953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.876877 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf\" (UID: \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" Sep 09 21:14:18.876953 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.876916 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f66kh\" (UniqueName: \"kubernetes.io/projected/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-kube-api-access-f66kh\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf\" (UID: \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" Sep 09 21:14:18.877277 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.877254 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf\" (UID: \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" Sep 09 21:14:18.877381 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.877296 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf\" (UID: \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" Sep 09 21:14:18.887689 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.887664 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f66kh\" (UniqueName: \"kubernetes.io/projected/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-kube-api-access-f66kh\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf\" (UID: \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" Sep 09 21:14:18.956538 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.956501 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" Sep 09 21:14:18.977704 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.977665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c074b89-8236-430e-a2f4-70a3921a956b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr\" (UID: \"3c074b89-8236-430e-a2f4-70a3921a956b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" Sep 09 21:14:18.977878 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.977726 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c074b89-8236-430e-a2f4-70a3921a956b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr\" (UID: \"3c074b89-8236-430e-a2f4-70a3921a956b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" Sep 09 21:14:18.977878 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.977768 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgnmn\" (UniqueName: \"kubernetes.io/projected/3c074b89-8236-430e-a2f4-70a3921a956b-kube-api-access-kgnmn\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr\" (UID: \"3c074b89-8236-430e-a2f4-70a3921a956b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" Sep 09 21:14:18.978153 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.978125 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c074b89-8236-430e-a2f4-70a3921a956b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr\" (UID: \"3c074b89-8236-430e-a2f4-70a3921a956b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" Sep 09 21:14:18.978239 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.978153 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c074b89-8236-430e-a2f4-70a3921a956b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr\" (UID: \"3c074b89-8236-430e-a2f4-70a3921a956b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" Sep 09 21:14:18.989613 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:18.989574 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgnmn\" (UniqueName: \"kubernetes.io/projected/3c074b89-8236-430e-a2f4-70a3921a956b-kube-api-access-kgnmn\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr\" (UID: \"3c074b89-8236-430e-a2f4-70a3921a956b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" Sep 09 21:14:19.059113 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.059078 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9"] Sep 09 21:14:19.060190 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:14:19.060158 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83e41dd2_a885_4441_9f5b_b9325a3aecaa.slice/crio-68080d3c2fc6487a9c1b0aa5db6a9b6f3b88c4380d1d3128d403efdbf4c9fa25 WatchSource:0}: Error finding container 68080d3c2fc6487a9c1b0aa5db6a9b6f3b88c4380d1d3128d403efdbf4c9fa25: Status 404 returned error can't find the container with id 68080d3c2fc6487a9c1b0aa5db6a9b6f3b88c4380d1d3128d403efdbf4c9fa25 Sep 09 21:14:19.109751 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.109725 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" Sep 09 21:14:19.139751 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.139718 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897"] Sep 09 21:14:19.141362 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:14:19.141325 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29f5f7a1_026b_4121_980e_20de06302bcb.slice/crio-485bfab31867d77da7955b789370b442597af3d970d19c6aa73ab93d56205119 WatchSource:0}: Error finding container 485bfab31867d77da7955b789370b442597af3d970d19c6aa73ab93d56205119: Status 404 returned error can't find the container with id 485bfab31867d77da7955b789370b442597af3d970d19c6aa73ab93d56205119 Sep 09 21:14:19.240271 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.240246 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf"] Sep 09 21:14:19.242590 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:14:19.242562 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df32d6f_ec04_4b4b_b08b_c944bb3f7f8d.slice/crio-e802765b19ffedd112788e7c24c3743f4abdef1726acf08ace6d687896a63e27 WatchSource:0}: Error finding container e802765b19ffedd112788e7c24c3743f4abdef1726acf08ace6d687896a63e27: Status 404 returned error can't find the container with id e802765b19ffedd112788e7c24c3743f4abdef1726acf08ace6d687896a63e27 Sep 09 21:14:19.404337 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.404300 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr"] Sep 09 21:14:19.407677 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:14:19.407643 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c074b89_8236_430e_a2f4_70a3921a956b.slice/crio-1de5cbb0be6d944bdf8626222facc920b5a70bddeb36b4090916995f9ea40503 WatchSource:0}: Error finding container 1de5cbb0be6d944bdf8626222facc920b5a70bddeb36b4090916995f9ea40503: Status 404 returned error can't find the container with id 1de5cbb0be6d944bdf8626222facc920b5a70bddeb36b4090916995f9ea40503 Sep 09 21:14:19.669296 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.669209 2560 generic.go:356] "Generic (PLEG): container finished" podID="3c074b89-8236-430e-a2f4-70a3921a956b" containerID="fda598e1036bb4f283b781f7b874994bf80516af0d48957023de65cffc6612e4" exitCode=0 Sep 09 21:14:19.669712 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.669304 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" event={"ID":"3c074b89-8236-430e-a2f4-70a3921a956b","Type":"ContainerDied","Data":"fda598e1036bb4f283b781f7b874994bf80516af0d48957023de65cffc6612e4"} Sep 09 21:14:19.669712 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.669362 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" event={"ID":"3c074b89-8236-430e-a2f4-70a3921a956b","Type":"ContainerStarted","Data":"1de5cbb0be6d944bdf8626222facc920b5a70bddeb36b4090916995f9ea40503"} Sep 09 21:14:19.670931 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.670910 2560 generic.go:356] "Generic (PLEG): container finished" podID="29f5f7a1-026b-4121-980e-20de06302bcb" containerID="b150b4a85774324d7943fa09f789c24dbcfac76ee563973708b59aecbdaf5e0f" exitCode=0 Sep 09 21:14:19.671034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.670939 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" event={"ID":"29f5f7a1-026b-4121-980e-20de06302bcb","Type":"ContainerDied","Data":"b150b4a85774324d7943fa09f789c24dbcfac76ee563973708b59aecbdaf5e0f"} Sep 09 21:14:19.671034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.670984 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" event={"ID":"29f5f7a1-026b-4121-980e-20de06302bcb","Type":"ContainerStarted","Data":"485bfab31867d77da7955b789370b442597af3d970d19c6aa73ab93d56205119"} Sep 09 21:14:19.672561 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.672545 2560 generic.go:356] "Generic (PLEG): container finished" podID="5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d" containerID="f508b1bdf908c3fcc6bf6f6c6042b76202c75ea026fe3173df2614c7e54cbe49" exitCode=0 Sep 09 21:14:19.672623 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.672602 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" event={"ID":"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d","Type":"ContainerDied","Data":"f508b1bdf908c3fcc6bf6f6c6042b76202c75ea026fe3173df2614c7e54cbe49"} Sep 09 21:14:19.672673 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.672640 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" event={"ID":"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d","Type":"ContainerStarted","Data":"e802765b19ffedd112788e7c24c3743f4abdef1726acf08ace6d687896a63e27"} Sep 09 21:14:19.674064 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.674032 2560 generic.go:356] "Generic (PLEG): container finished" podID="83e41dd2-a885-4441-9f5b-b9325a3aecaa" containerID="589c092dd78b03ad7c04e2b651ea3248a6a5d8d0ea60c68af0a473671a65b847" exitCode=0 Sep 09 21:14:19.674145 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.674074 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" event={"ID":"83e41dd2-a885-4441-9f5b-b9325a3aecaa","Type":"ContainerDied","Data":"589c092dd78b03ad7c04e2b651ea3248a6a5d8d0ea60c68af0a473671a65b847"} Sep 09 21:14:19.674145 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:19.674115 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" event={"ID":"83e41dd2-a885-4441-9f5b-b9325a3aecaa","Type":"ContainerStarted","Data":"68080d3c2fc6487a9c1b0aa5db6a9b6f3b88c4380d1d3128d403efdbf4c9fa25"} Sep 09 21:14:21.687511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:21.687422 2560 generic.go:356] "Generic (PLEG): container finished" podID="29f5f7a1-026b-4121-980e-20de06302bcb" containerID="9038a3afccbef6d9842536d6a780e8d8b8232be9738558cd780242f2b2e0a93f" exitCode=0 Sep 09 21:14:21.687915 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:21.687506 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" event={"ID":"29f5f7a1-026b-4121-980e-20de06302bcb","Type":"ContainerDied","Data":"9038a3afccbef6d9842536d6a780e8d8b8232be9738558cd780242f2b2e0a93f"} Sep 09 21:14:21.689512 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:21.689486 2560 generic.go:356] "Generic (PLEG): container finished" podID="5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d" containerID="6c67f5410d1071b594736db0d7947165e26adaa40437dacb5504715888c7937f" exitCode=0 Sep 09 21:14:21.690145 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:21.690069 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" event={"ID":"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d","Type":"ContainerDied","Data":"6c67f5410d1071b594736db0d7947165e26adaa40437dacb5504715888c7937f"} Sep 09 21:14:21.697303 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:21.697278 2560 generic.go:356] "Generic (PLEG): container finished" podID="83e41dd2-a885-4441-9f5b-b9325a3aecaa" containerID="a439b182bcfa03c9e6909a641d8f93802796a941c5d9281f02590d81f14bd7df" exitCode=0 Sep 09 21:14:21.697408 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:21.697317 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" event={"ID":"83e41dd2-a885-4441-9f5b-b9325a3aecaa","Type":"ContainerDied","Data":"a439b182bcfa03c9e6909a641d8f93802796a941c5d9281f02590d81f14bd7df"} Sep 09 21:14:21.699486 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:21.699464 2560 generic.go:356] "Generic (PLEG): container finished" podID="3c074b89-8236-430e-a2f4-70a3921a956b" containerID="cb4c793d8c6a992aa139cb3d82192f0a1ffd53f63ab770f630ae44aea8bf5b5b" exitCode=0 Sep 09 21:14:21.699558 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:21.699496 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" event={"ID":"3c074b89-8236-430e-a2f4-70a3921a956b","Type":"ContainerDied","Data":"cb4c793d8c6a992aa139cb3d82192f0a1ffd53f63ab770f630ae44aea8bf5b5b"} Sep 09 21:14:22.708098 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:22.708065 2560 generic.go:356] "Generic (PLEG): container finished" podID="5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d" containerID="5e2c45a819430f43dc9242d62faec54c64bc9d507bfce2c81c9bbd70399e0a9a" exitCode=0 Sep 09 21:14:22.708596 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:22.708134 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" event={"ID":"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d","Type":"ContainerDied","Data":"5e2c45a819430f43dc9242d62faec54c64bc9d507bfce2c81c9bbd70399e0a9a"} Sep 09 21:14:22.710342 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:22.710320 2560 generic.go:356] "Generic (PLEG): container finished" podID="83e41dd2-a885-4441-9f5b-b9325a3aecaa" containerID="8dade6ab9c20d6d56b98c4a0c64a3987edefb7ef83af0b3394e5770636edbdb1" exitCode=0 Sep 09 21:14:22.710456 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:22.710395 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" event={"ID":"83e41dd2-a885-4441-9f5b-b9325a3aecaa","Type":"ContainerDied","Data":"8dade6ab9c20d6d56b98c4a0c64a3987edefb7ef83af0b3394e5770636edbdb1"} Sep 09 21:14:22.712412 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:22.712393 2560 generic.go:356] "Generic (PLEG): container finished" podID="3c074b89-8236-430e-a2f4-70a3921a956b" containerID="6283c3623835c1695c547138486e4d4544e27df3459bbd6d10c06028e221d5e3" exitCode=0 Sep 09 21:14:22.712506 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:22.712469 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" event={"ID":"3c074b89-8236-430e-a2f4-70a3921a956b","Type":"ContainerDied","Data":"6283c3623835c1695c547138486e4d4544e27df3459bbd6d10c06028e221d5e3"} Sep 09 21:14:22.714500 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:22.714481 2560 generic.go:356] "Generic (PLEG): container finished" podID="29f5f7a1-026b-4121-980e-20de06302bcb" containerID="59906ebb51a6921e4960b223684ff17d10631d51800f012da81dfef72f060218" exitCode=0 Sep 09 21:14:22.714577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:22.714510 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" event={"ID":"29f5f7a1-026b-4121-980e-20de06302bcb","Type":"ContainerDied","Data":"59906ebb51a6921e4960b223684ff17d10631d51800f012da81dfef72f060218"} Sep 09 21:14:23.853530 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:23.853483 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" Sep 09 21:14:23.885337 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:23.885309 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" Sep 09 21:14:23.904061 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:23.904016 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" Sep 09 21:14:23.906920 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:23.906899 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" Sep 09 21:14:24.020083 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.019964 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f66kh\" (UniqueName: \"kubernetes.io/projected/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-kube-api-access-f66kh\") pod \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\" (UID: \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\") " Sep 09 21:14:24.020083 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.020017 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c074b89-8236-430e-a2f4-70a3921a956b-bundle\") pod \"3c074b89-8236-430e-a2f4-70a3921a956b\" (UID: \"3c074b89-8236-430e-a2f4-70a3921a956b\") " Sep 09 21:14:24.020083 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.020035 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c074b89-8236-430e-a2f4-70a3921a956b-util\") pod \"3c074b89-8236-430e-a2f4-70a3921a956b\" (UID: \"3c074b89-8236-430e-a2f4-70a3921a956b\") " Sep 09 21:14:24.020373 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.020090 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xv8jg\" (UniqueName: \"kubernetes.io/projected/83e41dd2-a885-4441-9f5b-b9325a3aecaa-kube-api-access-xv8jg\") pod \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\" (UID: \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\") " Sep 09 21:14:24.020373 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.020205 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-bundle\") pod \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\" (UID: \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\") " Sep 09 21:14:24.020373 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.020247 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29f5f7a1-026b-4121-980e-20de06302bcb-bundle\") pod \"29f5f7a1-026b-4121-980e-20de06302bcb\" (UID: \"29f5f7a1-026b-4121-980e-20de06302bcb\") " Sep 09 21:14:24.020373 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.020279 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83e41dd2-a885-4441-9f5b-b9325a3aecaa-util\") pod \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\" (UID: \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\") " Sep 09 21:14:24.020373 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.020306 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83e41dd2-a885-4441-9f5b-b9325a3aecaa-bundle\") pod \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\" (UID: \"83e41dd2-a885-4441-9f5b-b9325a3aecaa\") " Sep 09 21:14:24.020373 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.020330 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-util\") pod \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\" (UID: \"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d\") " Sep 09 21:14:24.020689 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.020380 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l5vj\" (UniqueName: \"kubernetes.io/projected/29f5f7a1-026b-4121-980e-20de06302bcb-kube-api-access-5l5vj\") pod \"29f5f7a1-026b-4121-980e-20de06302bcb\" (UID: \"29f5f7a1-026b-4121-980e-20de06302bcb\") " Sep 09 21:14:24.020689 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.020405 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgnmn\" (UniqueName: \"kubernetes.io/projected/3c074b89-8236-430e-a2f4-70a3921a956b-kube-api-access-kgnmn\") pod \"3c074b89-8236-430e-a2f4-70a3921a956b\" (UID: \"3c074b89-8236-430e-a2f4-70a3921a956b\") " Sep 09 21:14:24.020689 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.020452 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29f5f7a1-026b-4121-980e-20de06302bcb-util\") pod \"29f5f7a1-026b-4121-980e-20de06302bcb\" (UID: \"29f5f7a1-026b-4121-980e-20de06302bcb\") " Sep 09 21:14:24.021259 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.021225 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-bundle" (OuterVolumeSpecName: "bundle") pod "5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d" (UID: "5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:14:24.021649 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.021291 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c074b89-8236-430e-a2f4-70a3921a956b-bundle" (OuterVolumeSpecName: "bundle") pod "3c074b89-8236-430e-a2f4-70a3921a956b" (UID: "3c074b89-8236-430e-a2f4-70a3921a956b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:14:24.022226 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.022172 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e41dd2-a885-4441-9f5b-b9325a3aecaa-bundle" (OuterVolumeSpecName: "bundle") pod "83e41dd2-a885-4441-9f5b-b9325a3aecaa" (UID: "83e41dd2-a885-4441-9f5b-b9325a3aecaa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:14:24.022511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.022488 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29f5f7a1-026b-4121-980e-20de06302bcb-bundle" (OuterVolumeSpecName: "bundle") pod "29f5f7a1-026b-4121-980e-20de06302bcb" (UID: "29f5f7a1-026b-4121-980e-20de06302bcb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:14:24.022930 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.022904 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-kube-api-access-f66kh" (OuterVolumeSpecName: "kube-api-access-f66kh") pod "5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d" (UID: "5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d"). InnerVolumeSpecName "kube-api-access-f66kh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:14:24.023035 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.022998 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83e41dd2-a885-4441-9f5b-b9325a3aecaa-kube-api-access-xv8jg" (OuterVolumeSpecName: "kube-api-access-xv8jg") pod "83e41dd2-a885-4441-9f5b-b9325a3aecaa" (UID: "83e41dd2-a885-4441-9f5b-b9325a3aecaa"). InnerVolumeSpecName "kube-api-access-xv8jg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:14:24.024339 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.024316 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c074b89-8236-430e-a2f4-70a3921a956b-kube-api-access-kgnmn" (OuterVolumeSpecName: "kube-api-access-kgnmn") pod "3c074b89-8236-430e-a2f4-70a3921a956b" (UID: "3c074b89-8236-430e-a2f4-70a3921a956b"). InnerVolumeSpecName "kube-api-access-kgnmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:14:24.024449 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.024339 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29f5f7a1-026b-4121-980e-20de06302bcb-kube-api-access-5l5vj" (OuterVolumeSpecName: "kube-api-access-5l5vj") pod "29f5f7a1-026b-4121-980e-20de06302bcb" (UID: "29f5f7a1-026b-4121-980e-20de06302bcb"). InnerVolumeSpecName "kube-api-access-5l5vj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:14:24.026772 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.026741 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c074b89-8236-430e-a2f4-70a3921a956b-util" (OuterVolumeSpecName: "util") pod "3c074b89-8236-430e-a2f4-70a3921a956b" (UID: "3c074b89-8236-430e-a2f4-70a3921a956b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:14:24.027749 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.027713 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e41dd2-a885-4441-9f5b-b9325a3aecaa-util" (OuterVolumeSpecName: "util") pod "83e41dd2-a885-4441-9f5b-b9325a3aecaa" (UID: "83e41dd2-a885-4441-9f5b-b9325a3aecaa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:14:24.030308 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.030285 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29f5f7a1-026b-4121-980e-20de06302bcb-util" (OuterVolumeSpecName: "util") pod "29f5f7a1-026b-4121-980e-20de06302bcb" (UID: "29f5f7a1-026b-4121-980e-20de06302bcb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:14:24.030433 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.030418 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-util" (OuterVolumeSpecName: "util") pod "5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d" (UID: "5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:14:24.121800 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.121761 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l5vj\" (UniqueName: \"kubernetes.io/projected/29f5f7a1-026b-4121-980e-20de06302bcb-kube-api-access-5l5vj\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:24.121800 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.121795 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgnmn\" (UniqueName: \"kubernetes.io/projected/3c074b89-8236-430e-a2f4-70a3921a956b-kube-api-access-kgnmn\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:24.121800 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.121805 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29f5f7a1-026b-4121-980e-20de06302bcb-util\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:24.122022 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.121817 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f66kh\" (UniqueName: \"kubernetes.io/projected/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-kube-api-access-f66kh\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:24.122022 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.121826 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c074b89-8236-430e-a2f4-70a3921a956b-bundle\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:24.122022 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.121834 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c074b89-8236-430e-a2f4-70a3921a956b-util\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:24.122022 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.121842 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv8jg\" (UniqueName: \"kubernetes.io/projected/83e41dd2-a885-4441-9f5b-b9325a3aecaa-kube-api-access-xv8jg\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:24.122022 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.121850 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-bundle\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:24.122022 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.121859 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29f5f7a1-026b-4121-980e-20de06302bcb-bundle\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:24.122022 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.121866 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83e41dd2-a885-4441-9f5b-b9325a3aecaa-util\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:24.122022 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.121873 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83e41dd2-a885-4441-9f5b-b9325a3aecaa-bundle\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:24.122022 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.121880 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d-util\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:24.724867 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.724833 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" Sep 09 21:14:24.724867 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.724847 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vz897" event={"ID":"29f5f7a1-026b-4121-980e-20de06302bcb","Type":"ContainerDied","Data":"485bfab31867d77da7955b789370b442597af3d970d19c6aa73ab93d56205119"} Sep 09 21:14:24.725136 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.724918 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="485bfab31867d77da7955b789370b442597af3d970d19c6aa73ab93d56205119" Sep 09 21:14:24.726731 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.726715 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" Sep 09 21:14:24.726850 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.726715 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj89nf" event={"ID":"5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d","Type":"ContainerDied","Data":"e802765b19ffedd112788e7c24c3743f4abdef1726acf08ace6d687896a63e27"} Sep 09 21:14:24.726850 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.726829 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e802765b19ffedd112788e7c24c3743f4abdef1726acf08ace6d687896a63e27" Sep 09 21:14:24.728833 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.728814 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" Sep 09 21:14:24.728935 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.728833 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl9jv9" event={"ID":"83e41dd2-a885-4441-9f5b-b9325a3aecaa","Type":"ContainerDied","Data":"68080d3c2fc6487a9c1b0aa5db6a9b6f3b88c4380d1d3128d403efdbf4c9fa25"} Sep 09 21:14:24.728935 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.728872 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="68080d3c2fc6487a9c1b0aa5db6a9b6f3b88c4380d1d3128d403efdbf4c9fa25" Sep 09 21:14:24.730970 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.730949 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" event={"ID":"3c074b89-8236-430e-a2f4-70a3921a956b","Type":"ContainerDied","Data":"1de5cbb0be6d944bdf8626222facc920b5a70bddeb36b4090916995f9ea40503"} Sep 09 21:14:24.731080 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.730987 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1de5cbb0be6d944bdf8626222facc920b5a70bddeb36b4090916995f9ea40503" Sep 09 21:14:24.731080 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:24.730991 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c694bfr" Sep 09 21:14:27.842235 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:27.842185 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:29.915851 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.915812 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t"] Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916097 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83e41dd2-a885-4441-9f5b-b9325a3aecaa" containerName="pull" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916108 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="83e41dd2-a885-4441-9f5b-b9325a3aecaa" containerName="pull" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916118 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c074b89-8236-430e-a2f4-70a3921a956b" containerName="pull" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916124 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c074b89-8236-430e-a2f4-70a3921a956b" containerName="pull" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916130 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83e41dd2-a885-4441-9f5b-b9325a3aecaa" containerName="extract" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916135 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="83e41dd2-a885-4441-9f5b-b9325a3aecaa" containerName="extract" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916144 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d" containerName="util" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916149 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d" containerName="util" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916155 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29f5f7a1-026b-4121-980e-20de06302bcb" containerName="pull" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916160 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="29f5f7a1-026b-4121-980e-20de06302bcb" containerName="pull" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916166 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d" containerName="pull" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916171 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d" containerName="pull" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916178 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c074b89-8236-430e-a2f4-70a3921a956b" containerName="util" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916183 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c074b89-8236-430e-a2f4-70a3921a956b" containerName="util" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916192 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29f5f7a1-026b-4121-980e-20de06302bcb" containerName="util" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916197 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="29f5f7a1-026b-4121-980e-20de06302bcb" containerName="util" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916202 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29f5f7a1-026b-4121-980e-20de06302bcb" containerName="extract" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916206 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="29f5f7a1-026b-4121-980e-20de06302bcb" containerName="extract" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916213 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d" containerName="extract" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916218 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d" containerName="extract" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916227 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83e41dd2-a885-4441-9f5b-b9325a3aecaa" containerName="util" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916232 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="83e41dd2-a885-4441-9f5b-b9325a3aecaa" containerName="util" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916238 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c074b89-8236-430e-a2f4-70a3921a956b" containerName="extract" Sep 09 21:14:29.916260 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916243 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c074b89-8236-430e-a2f4-70a3921a956b" containerName="extract" Sep 09 21:14:29.917172 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916286 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5df32d6f-ec04-4b4b-b08b-c944bb3f7f8d" containerName="extract" Sep 09 21:14:29.917172 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916295 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="83e41dd2-a885-4441-9f5b-b9325a3aecaa" containerName="extract" Sep 09 21:14:29.917172 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916300 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="29f5f7a1-026b-4121-980e-20de06302bcb" containerName="extract" Sep 09 21:14:29.917172 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.916306 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c074b89-8236-430e-a2f4-70a3921a956b" containerName="extract" Sep 09 21:14:29.918986 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.918969 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:29.924296 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.924268 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 09 21:14:29.924447 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.924307 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 09 21:14:29.925626 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.925293 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6p6qg\"" Sep 09 21:14:29.934601 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:29.934575 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t"] Sep 09 21:14:30.073434 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:30.073394 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg8s2\" (UniqueName: \"kubernetes.io/projected/417f73db-05eb-4b1c-ba4e-de03f5bd6585-kube-api-access-bg8s2\") pod \"limitador-operator-controller-manager-74495c69f8-2tg8t\" (UID: \"417f73db-05eb-4b1c-ba4e-de03f5bd6585\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:30.174845 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:30.174752 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg8s2\" (UniqueName: \"kubernetes.io/projected/417f73db-05eb-4b1c-ba4e-de03f5bd6585-kube-api-access-bg8s2\") pod \"limitador-operator-controller-manager-74495c69f8-2tg8t\" (UID: \"417f73db-05eb-4b1c-ba4e-de03f5bd6585\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:30.185228 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:30.185198 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg8s2\" (UniqueName: \"kubernetes.io/projected/417f73db-05eb-4b1c-ba4e-de03f5bd6585-kube-api-access-bg8s2\") pod \"limitador-operator-controller-manager-74495c69f8-2tg8t\" (UID: \"417f73db-05eb-4b1c-ba4e-de03f5bd6585\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:30.229280 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:30.229237 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:30.506400 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:30.506375 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t"] Sep 09 21:14:30.508659 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:14:30.508625 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod417f73db_05eb_4b1c_ba4e_de03f5bd6585.slice/crio-42b041eb90c7653c74f23e156532644427c866a3aa39ebc7118f57770fb51d1d WatchSource:0}: Error finding container 42b041eb90c7653c74f23e156532644427c866a3aa39ebc7118f57770fb51d1d: Status 404 returned error can't find the container with id 42b041eb90c7653c74f23e156532644427c866a3aa39ebc7118f57770fb51d1d Sep 09 21:14:30.661736 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:30.661696 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:30.763542 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:30.763455 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" event={"ID":"417f73db-05eb-4b1c-ba4e-de03f5bd6585","Type":"ContainerStarted","Data":"42b041eb90c7653c74f23e156532644427c866a3aa39ebc7118f57770fb51d1d"} Sep 09 21:14:32.119538 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:32.119499 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-pkcxf"] Sep 09 21:14:32.124582 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:32.124552 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-pkcxf" Sep 09 21:14:32.128779 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:32.128754 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 09 21:14:32.129084 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:32.129063 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-nlt4t\"" Sep 09 21:14:32.145494 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:32.145464 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-pkcxf"] Sep 09 21:14:32.294888 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:32.294851 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jqwx\" (UniqueName: \"kubernetes.io/projected/99925480-9343-4aaa-990e-75c2b56399a0-kube-api-access-5jqwx\") pod \"dns-operator-controller-manager-7b84cd7c9c-pkcxf\" (UID: \"99925480-9343-4aaa-990e-75c2b56399a0\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-pkcxf" Sep 09 21:14:32.395718 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:32.395626 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jqwx\" (UniqueName: \"kubernetes.io/projected/99925480-9343-4aaa-990e-75c2b56399a0-kube-api-access-5jqwx\") pod \"dns-operator-controller-manager-7b84cd7c9c-pkcxf\" (UID: \"99925480-9343-4aaa-990e-75c2b56399a0\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-pkcxf" Sep 09 21:14:32.406148 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:32.406104 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jqwx\" (UniqueName: \"kubernetes.io/projected/99925480-9343-4aaa-990e-75c2b56399a0-kube-api-access-5jqwx\") pod \"dns-operator-controller-manager-7b84cd7c9c-pkcxf\" (UID: \"99925480-9343-4aaa-990e-75c2b56399a0\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-pkcxf" Sep 09 21:14:32.428198 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:32.428163 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:32.436399 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:32.436373 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-pkcxf" Sep 09 21:14:32.837622 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:32.837545 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:33.152484 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:33.152448 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:33.265122 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:33.265091 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:33.265443 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:33.265424 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-pkcxf"] Sep 09 21:14:33.267802 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:14:33.267776 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99925480_9343_4aaa_990e_75c2b56399a0.slice/crio-e5f81053690a4c8815c96ae04d9196846c49d6a0f7333082eeb65240c9127904 WatchSource:0}: Error finding container e5f81053690a4c8815c96ae04d9196846c49d6a0f7333082eeb65240c9127904: Status 404 returned error can't find the container with id e5f81053690a4c8815c96ae04d9196846c49d6a0f7333082eeb65240c9127904 Sep 09 21:14:33.783050 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:33.783002 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-pkcxf" event={"ID":"99925480-9343-4aaa-990e-75c2b56399a0","Type":"ContainerStarted","Data":"e5f81053690a4c8815c96ae04d9196846c49d6a0f7333082eeb65240c9127904"} Sep 09 21:14:33.784705 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:33.784679 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" event={"ID":"417f73db-05eb-4b1c-ba4e-de03f5bd6585","Type":"ContainerStarted","Data":"59c16a9ea89092ca75e21a860006b33f4703f3f884cf1f5e67d847f087b0e33c"} Sep 09 21:14:33.788063 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:33.785707 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:34.309174 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:34.309130 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:35.308153 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:35.308109 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:35.419533 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:35.419496 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:35.797526 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:35.797489 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-pkcxf" event={"ID":"99925480-9343-4aaa-990e-75c2b56399a0","Type":"ContainerStarted","Data":"6b668a352b01f5d956d4f6b0d1c3550c2cc98ec6028b0a3f0307f7d05521505c"} Sep 09 21:14:35.797774 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:35.797755 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-pkcxf" Sep 09 21:14:35.850549 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:35.850494 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" podStartSLOduration=4.325479954 podStartE2EDuration="6.850476527s" totalImagesPullingTime="2.524996573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:33.816610951 +0000 UTC m=+789.678133347" watchObservedRunningTime="2026-09-09 21:14:35.850476527 +0000 UTC m=+791.711998922" Sep 09 21:14:35.851250 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:35.851211 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-pkcxf" podStartSLOduration=2.212580348 podStartE2EDuration="3.851198615s" totalImagesPullingTime="1.638618267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:35.849628186 +0000 UTC m=+791.711150584" watchObservedRunningTime="2026-09-09 21:14:35.851198615 +0000 UTC m=+791.712721011" Sep 09 21:14:36.589024 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:36.588986 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc"] Sep 09 21:14:36.591233 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:36.591212 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:36.594983 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:36.594960 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-glswz\"" Sep 09 21:14:36.612889 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:36.612861 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc"] Sep 09 21:14:36.643757 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:36.643725 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:36.731502 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:36.731467 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb8e9def-aa5b-42d1-81da-efb2850c06eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" (UID: \"fb8e9def-aa5b-42d1-81da-efb2850c06eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:36.731642 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:36.731515 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc8fk\" (UniqueName: \"kubernetes.io/projected/fb8e9def-aa5b-42d1-81da-efb2850c06eb-kube-api-access-jc8fk\") pod \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" (UID: \"fb8e9def-aa5b-42d1-81da-efb2850c06eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:36.832213 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:36.832179 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc8fk\" (UniqueName: \"kubernetes.io/projected/fb8e9def-aa5b-42d1-81da-efb2850c06eb-kube-api-access-jc8fk\") pod \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" (UID: \"fb8e9def-aa5b-42d1-81da-efb2850c06eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:36.832382 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:36.832261 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb8e9def-aa5b-42d1-81da-efb2850c06eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" (UID: \"fb8e9def-aa5b-42d1-81da-efb2850c06eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:36.832612 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:36.832594 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb8e9def-aa5b-42d1-81da-efb2850c06eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" (UID: \"fb8e9def-aa5b-42d1-81da-efb2850c06eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:36.847889 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:36.847825 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc8fk\" (UniqueName: \"kubernetes.io/projected/fb8e9def-aa5b-42d1-81da-efb2850c06eb-kube-api-access-jc8fk\") pod \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" (UID: \"fb8e9def-aa5b-42d1-81da-efb2850c06eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:36.902383 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:36.902350 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:36.945960 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:36.945921 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:37.052431 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:37.052378 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:37.170353 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:37.170271 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:37.187734 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:37.187711 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc"] Sep 09 21:14:37.188870 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:14:37.188846 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb8e9def_aa5b_42d1_81da_efb2850c06eb.slice/crio-5ff66830c33d0751f35d8a530f5b85e4b7671aa78ded44138bf94f95efef4673 WatchSource:0}: Error finding container 5ff66830c33d0751f35d8a530f5b85e4b7671aa78ded44138bf94f95efef4673: Status 404 returned error can't find the container with id 5ff66830c33d0751f35d8a530f5b85e4b7671aa78ded44138bf94f95efef4673 Sep 09 21:14:37.816733 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:37.816693 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" event={"ID":"fb8e9def-aa5b-42d1-81da-efb2850c06eb","Type":"ContainerStarted","Data":"5ff66830c33d0751f35d8a530f5b85e4b7671aa78ded44138bf94f95efef4673"} Sep 09 21:14:39.267974 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:39.267933 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-8kdnh"] Sep 09 21:14:39.270672 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:39.270650 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-8kdnh" Sep 09 21:14:39.274124 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:39.274100 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-swcff\"" Sep 09 21:14:39.287940 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:39.287914 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-8kdnh"] Sep 09 21:14:39.457006 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:39.456968 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxgzz\" (UniqueName: \"kubernetes.io/projected/e1c77a06-4045-4d55-a5d3-0a50b1f3d8ec-kube-api-access-mxgzz\") pod \"authorino-operator-76b478bd78-8kdnh\" (UID: \"e1c77a06-4045-4d55-a5d3-0a50b1f3d8ec\") " pod="kuadrant-system/authorino-operator-76b478bd78-8kdnh" Sep 09 21:14:39.558368 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:39.558267 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxgzz\" (UniqueName: \"kubernetes.io/projected/e1c77a06-4045-4d55-a5d3-0a50b1f3d8ec-kube-api-access-mxgzz\") pod \"authorino-operator-76b478bd78-8kdnh\" (UID: \"e1c77a06-4045-4d55-a5d3-0a50b1f3d8ec\") " pod="kuadrant-system/authorino-operator-76b478bd78-8kdnh" Sep 09 21:14:39.572641 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:39.572607 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxgzz\" (UniqueName: \"kubernetes.io/projected/e1c77a06-4045-4d55-a5d3-0a50b1f3d8ec-kube-api-access-mxgzz\") pod \"authorino-operator-76b478bd78-8kdnh\" (UID: \"e1c77a06-4045-4d55-a5d3-0a50b1f3d8ec\") " pod="kuadrant-system/authorino-operator-76b478bd78-8kdnh" Sep 09 21:14:39.583336 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:39.583309 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-8kdnh" Sep 09 21:14:39.924005 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:39.923974 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-8kdnh"] Sep 09 21:14:40.448542 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:40.448501 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:40.548450 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:40.548393 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:40.878083 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:14:40.878028 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c77a06_4045_4d55_a5d3_0a50b1f3d8ec.slice/crio-67f89eb85a2b388f5280f998b9686b5bcb9f598c29bc1a74bf7bdc324493d2e8 WatchSource:0}: Error finding container 67f89eb85a2b388f5280f998b9686b5bcb9f598c29bc1a74bf7bdc324493d2e8: Status 404 returned error can't find the container with id 67f89eb85a2b388f5280f998b9686b5bcb9f598c29bc1a74bf7bdc324493d2e8 Sep 09 21:14:41.845316 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:41.845277 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-8kdnh" event={"ID":"e1c77a06-4045-4d55-a5d3-0a50b1f3d8ec","Type":"ContainerStarted","Data":"67f89eb85a2b388f5280f998b9686b5bcb9f598c29bc1a74bf7bdc324493d2e8"} Sep 09 21:14:41.847156 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:41.847127 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" event={"ID":"fb8e9def-aa5b-42d1-81da-efb2850c06eb","Type":"ContainerStarted","Data":"2f03f0699f32667fc1b75145438208063b06fd995ee5b91959b4f1811ce74356"} Sep 09 21:14:41.847277 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:41.847260 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:41.886852 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:41.886798 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" podStartSLOduration=2.150527043 podStartE2EDuration="5.886777668s" totalImagesPullingTime="3.736250625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:41.883870783 +0000 UTC m=+797.745393179" watchObservedRunningTime="2026-09-09 21:14:41.886777668 +0000 UTC m=+797.748300064" Sep 09 21:14:42.859720 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:42.859666 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-8kdnh" event={"ID":"e1c77a06-4045-4d55-a5d3-0a50b1f3d8ec","Type":"ContainerStarted","Data":"d2a7453c120e4e96f4d9e87dec8094e26b40e4a5ac79140ffffa6a83c43f6614"} Sep 09 21:14:42.886261 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:42.886213 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-8kdnh" podStartSLOduration=2.468182006 podStartE2EDuration="3.886197031s" totalImagesPullingTime="1.418015025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:42.883665715 +0000 UTC m=+798.745188120" watchObservedRunningTime="2026-09-09 21:14:42.886197031 +0000 UTC m=+798.747719425" Sep 09 21:14:43.442456 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.442420 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:43.543542 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.543501 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:43.804949 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.804905 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:43.828679 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.828648 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l"] Sep 09 21:14:43.830990 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.830973 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l" Sep 09 21:14:43.833774 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.833744 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Sep 09 21:14:43.834141 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.834124 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kgh6p\"" Sep 09 21:14:43.834356 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.834338 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Sep 09 21:14:43.845345 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.845312 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l"] Sep 09 21:14:43.865413 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.865373 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-8kdnh" Sep 09 21:14:43.888878 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.888846 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9967ce24-503f-4981-b9ee-c71a80beaadd-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-zgg7l\" (UID: \"9967ce24-503f-4981-b9ee-c71a80beaadd\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l" Sep 09 21:14:43.889082 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.888911 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbpvl\" (UniqueName: \"kubernetes.io/projected/9967ce24-503f-4981-b9ee-c71a80beaadd-kube-api-access-kbpvl\") pod \"kuadrant-console-plugin-fbdd4f445-zgg7l\" (UID: \"9967ce24-503f-4981-b9ee-c71a80beaadd\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l" Sep 09 21:14:43.889082 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.888990 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9967ce24-503f-4981-b9ee-c71a80beaadd-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-zgg7l\" (UID: \"9967ce24-503f-4981-b9ee-c71a80beaadd\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l" Sep 09 21:14:43.931153 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.931122 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:43.989985 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.989935 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9967ce24-503f-4981-b9ee-c71a80beaadd-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-zgg7l\" (UID: \"9967ce24-503f-4981-b9ee-c71a80beaadd\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l" Sep 09 21:14:43.989985 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.989989 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbpvl\" (UniqueName: \"kubernetes.io/projected/9967ce24-503f-4981-b9ee-c71a80beaadd-kube-api-access-kbpvl\") pod \"kuadrant-console-plugin-fbdd4f445-zgg7l\" (UID: \"9967ce24-503f-4981-b9ee-c71a80beaadd\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l" Sep 09 21:14:43.990206 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.990024 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9967ce24-503f-4981-b9ee-c71a80beaadd-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-zgg7l\" (UID: \"9967ce24-503f-4981-b9ee-c71a80beaadd\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l" Sep 09 21:14:43.990573 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.990552 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9967ce24-503f-4981-b9ee-c71a80beaadd-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-zgg7l\" (UID: \"9967ce24-503f-4981-b9ee-c71a80beaadd\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l" Sep 09 21:14:43.992460 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:43.992438 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9967ce24-503f-4981-b9ee-c71a80beaadd-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-zgg7l\" (UID: \"9967ce24-503f-4981-b9ee-c71a80beaadd\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l" Sep 09 21:14:44.007201 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:44.007174 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbpvl\" (UniqueName: \"kubernetes.io/projected/9967ce24-503f-4981-b9ee-c71a80beaadd-kube-api-access-kbpvl\") pod \"kuadrant-console-plugin-fbdd4f445-zgg7l\" (UID: \"9967ce24-503f-4981-b9ee-c71a80beaadd\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l" Sep 09 21:14:44.140017 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:44.139911 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l" Sep 09 21:14:44.413100 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:44.413076 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l"] Sep 09 21:14:44.414794 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:14:44.414765 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9967ce24_503f_4981_b9ee_c71a80beaadd.slice/crio-59b2bc622814e59d24790f212090d398e3cc30e584f36610a8d44a6b9481e3a4 WatchSource:0}: Error finding container 59b2bc622814e59d24790f212090d398e3cc30e584f36610a8d44a6b9481e3a4: Status 404 returned error can't find the container with id 59b2bc622814e59d24790f212090d398e3cc30e584f36610a8d44a6b9481e3a4 Sep 09 21:14:44.870767 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:44.870730 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l" event={"ID":"9967ce24-503f-4981-b9ee-c71a80beaadd","Type":"ContainerStarted","Data":"59b2bc622814e59d24790f212090d398e3cc30e584f36610a8d44a6b9481e3a4"} Sep 09 21:14:45.800174 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:45.800141 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:46.804859 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:46.804774 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-pkcxf" Sep 09 21:14:48.451964 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:48.451925 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:48.551826 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:48.551789 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:48.650586 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:48.650549 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:49.902641 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:49.902550 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l" event={"ID":"9967ce24-503f-4981-b9ee-c71a80beaadd","Type":"ContainerStarted","Data":"08ed2fdcc44959ab46ddffd98e5dc9f5d707603ec40d84df78c225782b3e5882"} Sep 09 21:14:49.925509 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:49.925458 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgg7l" podStartSLOduration=1.84434455 podStartE2EDuration="6.925441663s" totalImagesPullingTime="5.081097113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:49.924648253 +0000 UTC m=+805.786170649" watchObservedRunningTime="2026-09-09 21:14:49.925441663 +0000 UTC m=+805.786964057" Sep 09 21:14:50.404410 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:50.404376 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:52.861853 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:52.861817 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:54.438676 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:54.438638 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc"] Sep 09 21:14:54.442207 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:54.442183 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:54.458074 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:54.458032 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc"] Sep 09 21:14:54.464655 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:54.464628 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njhfz\" (UniqueName: \"kubernetes.io/projected/d38be29f-2ee2-4738-a68f-98b493ddc32f-kube-api-access-njhfz\") pod \"kuadrant-operator-controller-manager-5f685c496f-h77lc\" (UID: \"d38be29f-2ee2-4738-a68f-98b493ddc32f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:54.464781 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:54.464682 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d38be29f-2ee2-4738-a68f-98b493ddc32f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-h77lc\" (UID: \"d38be29f-2ee2-4738-a68f-98b493ddc32f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:54.565281 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:54.565221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njhfz\" (UniqueName: \"kubernetes.io/projected/d38be29f-2ee2-4738-a68f-98b493ddc32f-kube-api-access-njhfz\") pod \"kuadrant-operator-controller-manager-5f685c496f-h77lc\" (UID: \"d38be29f-2ee2-4738-a68f-98b493ddc32f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:54.565516 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:54.565299 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d38be29f-2ee2-4738-a68f-98b493ddc32f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-h77lc\" (UID: \"d38be29f-2ee2-4738-a68f-98b493ddc32f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:54.565698 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:54.565675 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d38be29f-2ee2-4738-a68f-98b493ddc32f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-h77lc\" (UID: \"d38be29f-2ee2-4738-a68f-98b493ddc32f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:54.575139 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:54.575106 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njhfz\" (UniqueName: \"kubernetes.io/projected/d38be29f-2ee2-4738-a68f-98b493ddc32f-kube-api-access-njhfz\") pod \"kuadrant-operator-controller-manager-5f685c496f-h77lc\" (UID: \"d38be29f-2ee2-4738-a68f-98b493ddc32f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:54.755453 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:54.755411 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:54.873358 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:54.873330 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-8kdnh" Sep 09 21:14:55.038286 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.038263 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc"] Sep 09 21:14:55.040555 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:14:55.040531 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38be29f_2ee2_4738_a68f_98b493ddc32f.slice/crio-2558cafa02516002ecb5c6c4a4d52dec80b84fe849c4830299cae01483635aa1 WatchSource:0}: Error finding container 2558cafa02516002ecb5c6c4a4d52dec80b84fe849c4830299cae01483635aa1: Status 404 returned error can't find the container with id 2558cafa02516002ecb5c6c4a4d52dec80b84fe849c4830299cae01483635aa1 Sep 09 21:14:55.113265 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.113240 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc"] Sep 09 21:14:55.123370 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.123344 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc"] Sep 09 21:14:55.137014 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.136992 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc"] Sep 09 21:14:55.137207 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.137183 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" podUID="fb8e9def-aa5b-42d1-81da-efb2850c06eb" containerName="manager" containerID="cri-o://2f03f0699f32667fc1b75145438208063b06fd995ee5b91959b4f1811ce74356" gracePeriod=2 Sep 09 21:14:55.144453 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.144427 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8"] Sep 09 21:14:55.144906 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.144886 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38be29f-2ee2-4738-a68f-98b493ddc32f" containerName="manager" Sep 09 21:14:55.144906 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.144907 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38be29f-2ee2-4738-a68f-98b493ddc32f" containerName="manager" Sep 09 21:14:55.145086 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.144989 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d38be29f-2ee2-4738-a68f-98b493ddc32f" containerName="manager" Sep 09 21:14:55.148549 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.148528 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" Sep 09 21:14:55.152807 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.152784 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc"] Sep 09 21:14:55.153411 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.153377 2560 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"fb8e9def-aa5b-42d1-81da-efb2850c06eb\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\": pods \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:55.163204 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.162423 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8"] Sep 09 21:14:55.165531 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.165506 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t"] Sep 09 21:14:55.165707 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.165687 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" podUID="417f73db-05eb-4b1c-ba4e-de03f5bd6585" containerName="manager" containerID="cri-o://59c16a9ea89092ca75e21a860006b33f4703f3f884cf1f5e67d847f087b0e33c" gracePeriod=2 Sep 09 21:14:55.168660 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.168635 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd"] Sep 09 21:14:55.169178 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.169157 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb8e9def-aa5b-42d1-81da-efb2850c06eb" containerName="manager" Sep 09 21:14:55.169178 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.169180 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb8e9def-aa5b-42d1-81da-efb2850c06eb" containerName="manager" Sep 09 21:14:55.169321 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.169266 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb8e9def-aa5b-42d1-81da-efb2850c06eb" containerName="manager" Sep 09 21:14:55.169896 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.169718 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/897abd07-1df0-4e7f-aed0-be88b430bbf1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-qnlx8\" (UID: \"897abd07-1df0-4e7f-aed0-be88b430bbf1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" Sep 09 21:14:55.169896 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.169783 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfrmq\" (UniqueName: \"kubernetes.io/projected/897abd07-1df0-4e7f-aed0-be88b430bbf1-kube-api-access-tfrmq\") pod \"kuadrant-operator-controller-manager-5f685c496f-qnlx8\" (UID: \"897abd07-1df0-4e7f-aed0-be88b430bbf1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" Sep 09 21:14:55.172407 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.172389 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" Sep 09 21:14:55.176468 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.176426 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t"] Sep 09 21:14:55.180523 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.180494 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="fb8e9def-aa5b-42d1-81da-efb2850c06eb" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:55.191876 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.191848 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd"] Sep 09 21:14:55.206018 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.205991 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-76knv"] Sep 09 21:14:55.206360 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.206344 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="417f73db-05eb-4b1c-ba4e-de03f5bd6585" containerName="manager" Sep 09 21:14:55.206439 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.206362 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="417f73db-05eb-4b1c-ba4e-de03f5bd6585" containerName="manager" Sep 09 21:14:55.206496 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.206477 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="417f73db-05eb-4b1c-ba4e-de03f5bd6585" containerName="manager" Sep 09 21:14:55.209511 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.209492 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76knv" Sep 09 21:14:55.219364 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.219327 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2tg8t\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="417f73db-05eb-4b1c-ba4e-de03f5bd6585" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:55.221537 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.221502 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="fb8e9def-aa5b-42d1-81da-efb2850c06eb" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:55.222946 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.222922 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-76knv"] Sep 09 21:14:55.227412 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.227386 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2tg8t\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="417f73db-05eb-4b1c-ba4e-de03f5bd6585" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:55.229700 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.229675 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="fb8e9def-aa5b-42d1-81da-efb2850c06eb" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:55.271157 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.271128 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfrmq\" (UniqueName: \"kubernetes.io/projected/897abd07-1df0-4e7f-aed0-be88b430bbf1-kube-api-access-tfrmq\") pod \"kuadrant-operator-controller-manager-5f685c496f-qnlx8\" (UID: \"897abd07-1df0-4e7f-aed0-be88b430bbf1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" Sep 09 21:14:55.271285 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.271237 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqcm8\" (UniqueName: \"kubernetes.io/projected/8fdf2319-7997-41ea-832b-d17ea32708c3-kube-api-access-sqcm8\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hgcsd\" (UID: \"8fdf2319-7997-41ea-832b-d17ea32708c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" Sep 09 21:14:55.271344 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.271272 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt65b\" (UniqueName: \"kubernetes.io/projected/59118908-dd81-40c1-ba8c-6beff25aaa9f-kube-api-access-tt65b\") pod \"limitador-operator-controller-manager-74495c69f8-76knv\" (UID: \"59118908-dd81-40c1-ba8c-6beff25aaa9f\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76knv" Sep 09 21:14:55.271344 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.271312 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fdf2319-7997-41ea-832b-d17ea32708c3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hgcsd\" (UID: \"8fdf2319-7997-41ea-832b-d17ea32708c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" Sep 09 21:14:55.271448 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.271366 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/897abd07-1df0-4e7f-aed0-be88b430bbf1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-qnlx8\" (UID: \"897abd07-1df0-4e7f-aed0-be88b430bbf1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" Sep 09 21:14:55.271686 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.271670 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/897abd07-1df0-4e7f-aed0-be88b430bbf1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-qnlx8\" (UID: \"897abd07-1df0-4e7f-aed0-be88b430bbf1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" Sep 09 21:14:55.285363 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.285338 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfrmq\" (UniqueName: \"kubernetes.io/projected/897abd07-1df0-4e7f-aed0-be88b430bbf1-kube-api-access-tfrmq\") pod \"kuadrant-operator-controller-manager-5f685c496f-qnlx8\" (UID: \"897abd07-1df0-4e7f-aed0-be88b430bbf1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" Sep 09 21:14:55.372645 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.372602 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqcm8\" (UniqueName: \"kubernetes.io/projected/8fdf2319-7997-41ea-832b-d17ea32708c3-kube-api-access-sqcm8\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hgcsd\" (UID: \"8fdf2319-7997-41ea-832b-d17ea32708c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" Sep 09 21:14:55.372838 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.372659 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt65b\" (UniqueName: \"kubernetes.io/projected/59118908-dd81-40c1-ba8c-6beff25aaa9f-kube-api-access-tt65b\") pod \"limitador-operator-controller-manager-74495c69f8-76knv\" (UID: \"59118908-dd81-40c1-ba8c-6beff25aaa9f\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76knv" Sep 09 21:14:55.372838 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.372692 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fdf2319-7997-41ea-832b-d17ea32708c3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hgcsd\" (UID: \"8fdf2319-7997-41ea-832b-d17ea32708c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" Sep 09 21:14:55.373225 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.373193 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fdf2319-7997-41ea-832b-d17ea32708c3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hgcsd\" (UID: \"8fdf2319-7997-41ea-832b-d17ea32708c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" Sep 09 21:14:55.383466 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.383439 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt65b\" (UniqueName: \"kubernetes.io/projected/59118908-dd81-40c1-ba8c-6beff25aaa9f-kube-api-access-tt65b\") pod \"limitador-operator-controller-manager-74495c69f8-76knv\" (UID: \"59118908-dd81-40c1-ba8c-6beff25aaa9f\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76knv" Sep 09 21:14:55.383575 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.383493 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqcm8\" (UniqueName: \"kubernetes.io/projected/8fdf2319-7997-41ea-832b-d17ea32708c3-kube-api-access-sqcm8\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hgcsd\" (UID: \"8fdf2319-7997-41ea-832b-d17ea32708c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" Sep 09 21:14:55.390486 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.390468 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:55.393445 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.393419 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2tg8t\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="417f73db-05eb-4b1c-ba4e-de03f5bd6585" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:55.395712 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.395689 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="fb8e9def-aa5b-42d1-81da-efb2850c06eb" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:55.396989 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.396972 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:55.399271 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.399247 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2tg8t\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="417f73db-05eb-4b1c-ba4e-de03f5bd6585" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:55.401558 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.401536 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="fb8e9def-aa5b-42d1-81da-efb2850c06eb" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:55.473812 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.473773 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bg8s2\" (UniqueName: \"kubernetes.io/projected/417f73db-05eb-4b1c-ba4e-de03f5bd6585-kube-api-access-bg8s2\") pod \"417f73db-05eb-4b1c-ba4e-de03f5bd6585\" (UID: \"417f73db-05eb-4b1c-ba4e-de03f5bd6585\") " Sep 09 21:14:55.473812 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.473813 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jc8fk\" (UniqueName: \"kubernetes.io/projected/fb8e9def-aa5b-42d1-81da-efb2850c06eb-kube-api-access-jc8fk\") pod \"fb8e9def-aa5b-42d1-81da-efb2850c06eb\" (UID: \"fb8e9def-aa5b-42d1-81da-efb2850c06eb\") " Sep 09 21:14:55.474352 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.473846 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb8e9def-aa5b-42d1-81da-efb2850c06eb-extensions-socket-volume\") pod \"fb8e9def-aa5b-42d1-81da-efb2850c06eb\" (UID: \"fb8e9def-aa5b-42d1-81da-efb2850c06eb\") " Sep 09 21:14:55.474401 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.474366 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb8e9def-aa5b-42d1-81da-efb2850c06eb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fb8e9def-aa5b-42d1-81da-efb2850c06eb" (UID: "fb8e9def-aa5b-42d1-81da-efb2850c06eb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:14:55.476054 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.476017 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb8e9def-aa5b-42d1-81da-efb2850c06eb-kube-api-access-jc8fk" (OuterVolumeSpecName: "kube-api-access-jc8fk") pod "fb8e9def-aa5b-42d1-81da-efb2850c06eb" (UID: "fb8e9def-aa5b-42d1-81da-efb2850c06eb"). InnerVolumeSpecName "kube-api-access-jc8fk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:14:55.476140 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.476070 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/417f73db-05eb-4b1c-ba4e-de03f5bd6585-kube-api-access-bg8s2" (OuterVolumeSpecName: "kube-api-access-bg8s2") pod "417f73db-05eb-4b1c-ba4e-de03f5bd6585" (UID: "417f73db-05eb-4b1c-ba4e-de03f5bd6585"). InnerVolumeSpecName "kube-api-access-bg8s2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:14:55.552378 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.552340 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" Sep 09 21:14:55.565718 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.565689 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" Sep 09 21:14:55.571930 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.571902 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76knv" Sep 09 21:14:55.574923 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.574730 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bg8s2\" (UniqueName: \"kubernetes.io/projected/417f73db-05eb-4b1c-ba4e-de03f5bd6585-kube-api-access-bg8s2\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:55.574923 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.574760 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jc8fk\" (UniqueName: \"kubernetes.io/projected/fb8e9def-aa5b-42d1-81da-efb2850c06eb-kube-api-access-jc8fk\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:55.574923 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.574775 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb8e9def-aa5b-42d1-81da-efb2850c06eb-extensions-socket-volume\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:55.854778 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.854559 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8"] Sep 09 21:14:55.857451 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:14:55.857426 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod897abd07_1df0_4e7f_aed0_be88b430bbf1.slice/crio-ee3733dd5f6f9ebb6f0d3d9ea449085218c98941c2e4891ad492b3e8027111a0 WatchSource:0}: Error finding container ee3733dd5f6f9ebb6f0d3d9ea449085218c98941c2e4891ad492b3e8027111a0: Status 404 returned error can't find the container with id ee3733dd5f6f9ebb6f0d3d9ea449085218c98941c2e4891ad492b3e8027111a0 Sep 09 21:14:55.867559 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:14:55.867529 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fdf2319_7997_41ea_832b_d17ea32708c3.slice/crio-13a7dc8c95ec8c70680d805a427bc1d4e04ea65460a1c32d21064fbc95114802 WatchSource:0}: Error finding container 13a7dc8c95ec8c70680d805a427bc1d4e04ea65460a1c32d21064fbc95114802: Status 404 returned error can't find the container with id 13a7dc8c95ec8c70680d805a427bc1d4e04ea65460a1c32d21064fbc95114802 Sep 09 21:14:55.868176 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.868155 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd"] Sep 09 21:14:55.870319 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.870299 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-76knv"] Sep 09 21:14:55.874950 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:14:55.874923 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59118908_dd81_40c1_ba8c_6beff25aaa9f.slice/crio-c26393ed07f8f3053fa987b2f72cd5067d07dbeee81e8165f35d1a908ddb72d1 WatchSource:0}: Error finding container c26393ed07f8f3053fa987b2f72cd5067d07dbeee81e8165f35d1a908ddb72d1: Status 404 returned error can't find the container with id c26393ed07f8f3053fa987b2f72cd5067d07dbeee81e8165f35d1a908ddb72d1 Sep 09 21:14:55.936707 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.936678 2560 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" container="manager" Sep 09 21:14:55.938035 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.938007 2560 generic.go:356] "Generic (PLEG): container finished" podID="fb8e9def-aa5b-42d1-81da-efb2850c06eb" containerID="2f03f0699f32667fc1b75145438208063b06fd995ee5b91959b4f1811ce74356" exitCode=0 Sep 09 21:14:55.938158 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.938066 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:55.938158 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.938093 2560 scope.go:117] "RemoveContainer" containerID="2f03f0699f32667fc1b75145438208063b06fd995ee5b91959b4f1811ce74356" Sep 09 21:14:55.939222 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.939194 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-h77lc\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="d38be29f-2ee2-4738-a68f-98b493ddc32f" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:55.939899 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.939876 2560 generic.go:356] "Generic (PLEG): container finished" podID="417f73db-05eb-4b1c-ba4e-de03f5bd6585" containerID="59c16a9ea89092ca75e21a860006b33f4703f3f884cf1f5e67d847f087b0e33c" exitCode=0 Sep 09 21:14:55.940121 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.940026 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:55.941883 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.941845 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2tg8t\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="417f73db-05eb-4b1c-ba4e-de03f5bd6585" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:55.942763 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.942733 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" event={"ID":"897abd07-1df0-4e7f-aed0-be88b430bbf1","Type":"ContainerStarted","Data":"ef311284a960674b5eb2cd582f3b67100aab84bdda95b54dd9521a3548907647"} Sep 09 21:14:55.942856 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.942826 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" event={"ID":"897abd07-1df0-4e7f-aed0-be88b430bbf1","Type":"ContainerStarted","Data":"ee3733dd5f6f9ebb6f0d3d9ea449085218c98941c2e4891ad492b3e8027111a0"} Sep 09 21:14:55.943344 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.943323 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" Sep 09 21:14:55.944325 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.944237 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="fb8e9def-aa5b-42d1-81da-efb2850c06eb" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:55.945660 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.945628 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76knv" event={"ID":"59118908-dd81-40c1-ba8c-6beff25aaa9f","Type":"ContainerStarted","Data":"360ff0f2a72b2b3a3a677a6a7bea637ecb516ce03a4b32cd433a76b147b4b125"} Sep 09 21:14:55.945752 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.945684 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76knv" event={"ID":"59118908-dd81-40c1-ba8c-6beff25aaa9f","Type":"ContainerStarted","Data":"c26393ed07f8f3053fa987b2f72cd5067d07dbeee81e8165f35d1a908ddb72d1"} Sep 09 21:14:55.946475 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.946173 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76knv" Sep 09 21:14:55.946475 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.946356 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-h77lc\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="d38be29f-2ee2-4738-a68f-98b493ddc32f" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:55.951696 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.951577 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" event={"ID":"8fdf2319-7997-41ea-832b-d17ea32708c3","Type":"ContainerStarted","Data":"177faf00a88ea53a2882289c92d987bcba87a764c9de6ca0625e134c3947f8c5"} Sep 09 21:14:55.951696 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.951635 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" event={"ID":"8fdf2319-7997-41ea-832b-d17ea32708c3","Type":"ContainerStarted","Data":"13a7dc8c95ec8c70680d805a427bc1d4e04ea65460a1c32d21064fbc95114802"} Sep 09 21:14:55.952185 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.952114 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" Sep 09 21:14:55.953654 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.953621 2560 scope.go:117] "RemoveContainer" containerID="2f03f0699f32667fc1b75145438208063b06fd995ee5b91959b4f1811ce74356" Sep 09 21:14:55.954027 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.954005 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f03f0699f32667fc1b75145438208063b06fd995ee5b91959b4f1811ce74356\": container with ID starting with 2f03f0699f32667fc1b75145438208063b06fd995ee5b91959b4f1811ce74356 not found: ID does not exist" containerID="2f03f0699f32667fc1b75145438208063b06fd995ee5b91959b4f1811ce74356" Sep 09 21:14:55.954300 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.954276 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f03f0699f32667fc1b75145438208063b06fd995ee5b91959b4f1811ce74356"} err="failed to get container status \"2f03f0699f32667fc1b75145438208063b06fd995ee5b91959b4f1811ce74356\": rpc error: code = NotFound desc = could not find container \"2f03f0699f32667fc1b75145438208063b06fd995ee5b91959b4f1811ce74356\": container with ID starting with 2f03f0699f32667fc1b75145438208063b06fd995ee5b91959b4f1811ce74356 not found: ID does not exist" Sep 09 21:14:55.954408 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.954396 2560 scope.go:117] "RemoveContainer" containerID="59c16a9ea89092ca75e21a860006b33f4703f3f884cf1f5e67d847f087b0e33c" Sep 09 21:14:55.965573 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.965555 2560 scope.go:117] "RemoveContainer" containerID="59c16a9ea89092ca75e21a860006b33f4703f3f884cf1f5e67d847f087b0e33c" Sep 09 21:14:55.965873 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.965854 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59c16a9ea89092ca75e21a860006b33f4703f3f884cf1f5e67d847f087b0e33c\": container with ID starting with 59c16a9ea89092ca75e21a860006b33f4703f3f884cf1f5e67d847f087b0e33c not found: ID does not exist" containerID="59c16a9ea89092ca75e21a860006b33f4703f3f884cf1f5e67d847f087b0e33c" Sep 09 21:14:55.965922 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.965881 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59c16a9ea89092ca75e21a860006b33f4703f3f884cf1f5e67d847f087b0e33c"} err="failed to get container status \"59c16a9ea89092ca75e21a860006b33f4703f3f884cf1f5e67d847f087b0e33c\": rpc error: code = NotFound desc = could not find container \"59c16a9ea89092ca75e21a860006b33f4703f3f884cf1f5e67d847f087b0e33c\": container with ID starting with 59c16a9ea89092ca75e21a860006b33f4703f3f884cf1f5e67d847f087b0e33c not found: ID does not exist" Sep 09 21:14:55.973505 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:55.973462 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" podStartSLOduration=0.973447345 podStartE2EDuration="973.447345ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:55.971273656 +0000 UTC m=+811.832796051" watchObservedRunningTime="2026-09-09 21:14:55.973447345 +0000 UTC m=+811.834969734" Sep 09 21:14:55.973604 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.973486 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2tg8t\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="417f73db-05eb-4b1c-ba4e-de03f5bd6585" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:55.975653 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:55.975622 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="fb8e9def-aa5b-42d1-81da-efb2850c06eb" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:56.002164 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:56.002106 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" podStartSLOduration=1.002090337 podStartE2EDuration="1.002090337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:56.000813651 +0000 UTC m=+811.862336079" watchObservedRunningTime="2026-09-09 21:14:56.002090337 +0000 UTC m=+811.863612728" Sep 09 21:14:56.021793 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:56.021746 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2tg8t\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="417f73db-05eb-4b1c-ba4e-de03f5bd6585" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2tg8t" Sep 09 21:14:56.022002 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:56.021960 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76knv" podStartSLOduration=1.021944323 podStartE2EDuration="1.021944323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:56.01952736 +0000 UTC m=+811.881049755" watchObservedRunningTime="2026-09-09 21:14:56.021944323 +0000 UTC m=+811.883466718" Sep 09 21:14:56.023864 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:56.023843 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-p7dhc\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="fb8e9def-aa5b-42d1-81da-efb2850c06eb" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-p7dhc" Sep 09 21:14:56.025973 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:56.025946 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-h77lc\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="d38be29f-2ee2-4738-a68f-98b493ddc32f" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:56.780429 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:56.780396 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="417f73db-05eb-4b1c-ba4e-de03f5bd6585" path="/var/lib/kubelet/pods/417f73db-05eb-4b1c-ba4e-de03f5bd6585/volumes" Sep 09 21:14:56.780804 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:56.780754 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb8e9def-aa5b-42d1-81da-efb2850c06eb" path="/var/lib/kubelet/pods/fb8e9def-aa5b-42d1-81da-efb2850c06eb/volumes" Sep 09 21:14:56.960208 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:56.960169 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" podUID="d38be29f-2ee2-4738-a68f-98b493ddc32f" containerName="manager" containerID="cri-o://de143838affdbe1f291b320efb50c21a4ad3139062099ce6615cd3650a16d5ea" gracePeriod=2 Sep 09 21:14:57.212014 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:57.211987 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:57.214572 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:57.214548 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-h77lc\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="d38be29f-2ee2-4738-a68f-98b493ddc32f" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:57.288481 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:57.288444 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d38be29f-2ee2-4738-a68f-98b493ddc32f-extensions-socket-volume\") pod \"d38be29f-2ee2-4738-a68f-98b493ddc32f\" (UID: \"d38be29f-2ee2-4738-a68f-98b493ddc32f\") " Sep 09 21:14:57.288481 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:57.288485 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njhfz\" (UniqueName: \"kubernetes.io/projected/d38be29f-2ee2-4738-a68f-98b493ddc32f-kube-api-access-njhfz\") pod \"d38be29f-2ee2-4738-a68f-98b493ddc32f\" (UID: \"d38be29f-2ee2-4738-a68f-98b493ddc32f\") " Sep 09 21:14:57.288742 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:57.288721 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d38be29f-2ee2-4738-a68f-98b493ddc32f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d38be29f-2ee2-4738-a68f-98b493ddc32f" (UID: "d38be29f-2ee2-4738-a68f-98b493ddc32f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:14:57.290625 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:57.290590 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d38be29f-2ee2-4738-a68f-98b493ddc32f-kube-api-access-njhfz" (OuterVolumeSpecName: "kube-api-access-njhfz") pod "d38be29f-2ee2-4738-a68f-98b493ddc32f" (UID: "d38be29f-2ee2-4738-a68f-98b493ddc32f"). InnerVolumeSpecName "kube-api-access-njhfz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:14:57.297261 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:57.297232 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:57.389504 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:57.389422 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d38be29f-2ee2-4738-a68f-98b493ddc32f-extensions-socket-volume\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:57.389504 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:57.389451 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njhfz\" (UniqueName: \"kubernetes.io/projected/d38be29f-2ee2-4738-a68f-98b493ddc32f-kube-api-access-njhfz\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:14:57.399996 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:57.399966 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:57.969353 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:57.969320 2560 generic.go:356] "Generic (PLEG): container finished" podID="d38be29f-2ee2-4738-a68f-98b493ddc32f" containerID="de143838affdbe1f291b320efb50c21a4ad3139062099ce6615cd3650a16d5ea" exitCode=2 Sep 09 21:14:57.969763 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:57.969372 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:57.969763 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:57.969424 2560 scope.go:117] "RemoveContainer" containerID="de143838affdbe1f291b320efb50c21a4ad3139062099ce6615cd3650a16d5ea" Sep 09 21:14:57.971779 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:57.971757 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-h77lc\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="d38be29f-2ee2-4738-a68f-98b493ddc32f" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:57.978712 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:57.978694 2560 scope.go:117] "RemoveContainer" containerID="de143838affdbe1f291b320efb50c21a4ad3139062099ce6615cd3650a16d5ea" Sep 09 21:14:57.978983 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:57.978965 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de143838affdbe1f291b320efb50c21a4ad3139062099ce6615cd3650a16d5ea\": container with ID starting with de143838affdbe1f291b320efb50c21a4ad3139062099ce6615cd3650a16d5ea not found: ID does not exist" containerID="de143838affdbe1f291b320efb50c21a4ad3139062099ce6615cd3650a16d5ea" Sep 09 21:14:57.979075 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:57.978992 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de143838affdbe1f291b320efb50c21a4ad3139062099ce6615cd3650a16d5ea"} err="failed to get container status \"de143838affdbe1f291b320efb50c21a4ad3139062099ce6615cd3650a16d5ea\": rpc error: code = NotFound desc = could not find container \"de143838affdbe1f291b320efb50c21a4ad3139062099ce6615cd3650a16d5ea\": container with ID starting with de143838affdbe1f291b320efb50c21a4ad3139062099ce6615cd3650a16d5ea not found: ID does not exist" Sep 09 21:14:57.983210 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:14:57.983183 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-h77lc\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="d38be29f-2ee2-4738-a68f-98b493ddc32f" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-h77lc" Sep 09 21:14:58.780007 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:58.779971 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d38be29f-2ee2-4738-a68f-98b493ddc32f" path="/var/lib/kubelet/pods/d38be29f-2ee2-4738-a68f-98b493ddc32f/volumes" Sep 09 21:14:59.884077 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:59.884028 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:14:59.986201 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:14:59.986167 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:00.586265 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:00.586231 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:00.686796 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:00.686744 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:02.891868 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:02.891819 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:03.015665 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:03.015624 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:06.965104 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:06.965063 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" Sep 09 21:15:07.972010 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:07.971978 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" Sep 09 21:15:07.972416 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:07.972072 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-76knv" Sep 09 21:15:08.044606 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.044576 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd"] Sep 09 21:15:08.044783 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.044757 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" podUID="8fdf2319-7997-41ea-832b-d17ea32708c3" containerName="manager" containerID="cri-o://177faf00a88ea53a2882289c92d987bcba87a764c9de6ca0625e134c3947f8c5" gracePeriod=10 Sep 09 21:15:08.287882 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.287855 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" Sep 09 21:15:08.378678 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.378646 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fdf2319-7997-41ea-832b-d17ea32708c3-extensions-socket-volume\") pod \"8fdf2319-7997-41ea-832b-d17ea32708c3\" (UID: \"8fdf2319-7997-41ea-832b-d17ea32708c3\") " Sep 09 21:15:08.378862 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.378704 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqcm8\" (UniqueName: \"kubernetes.io/projected/8fdf2319-7997-41ea-832b-d17ea32708c3-kube-api-access-sqcm8\") pod \"8fdf2319-7997-41ea-832b-d17ea32708c3\" (UID: \"8fdf2319-7997-41ea-832b-d17ea32708c3\") " Sep 09 21:15:08.379080 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.379035 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fdf2319-7997-41ea-832b-d17ea32708c3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8fdf2319-7997-41ea-832b-d17ea32708c3" (UID: "8fdf2319-7997-41ea-832b-d17ea32708c3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:15:08.380810 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.380776 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fdf2319-7997-41ea-832b-d17ea32708c3-kube-api-access-sqcm8" (OuterVolumeSpecName: "kube-api-access-sqcm8") pod "8fdf2319-7997-41ea-832b-d17ea32708c3" (UID: "8fdf2319-7997-41ea-832b-d17ea32708c3"). InnerVolumeSpecName "kube-api-access-sqcm8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:15:08.399467 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.399437 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd"] Sep 09 21:15:08.399751 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.399739 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fdf2319-7997-41ea-832b-d17ea32708c3" containerName="manager" Sep 09 21:15:08.399803 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.399753 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fdf2319-7997-41ea-832b-d17ea32708c3" containerName="manager" Sep 09 21:15:08.399842 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.399815 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fdf2319-7997-41ea-832b-d17ea32708c3" containerName="manager" Sep 09 21:15:08.402114 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.402098 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd" Sep 09 21:15:08.426623 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.426589 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd"] Sep 09 21:15:08.479259 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.479222 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89252b48-4768-494e-826d-7f9ddd24bb0b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gbmgd\" (UID: \"89252b48-4768-494e-826d-7f9ddd24bb0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd" Sep 09 21:15:08.479422 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.479316 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvvjj\" (UniqueName: \"kubernetes.io/projected/89252b48-4768-494e-826d-7f9ddd24bb0b-kube-api-access-jvvjj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gbmgd\" (UID: \"89252b48-4768-494e-826d-7f9ddd24bb0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd" Sep 09 21:15:08.479422 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.479361 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqcm8\" (UniqueName: \"kubernetes.io/projected/8fdf2319-7997-41ea-832b-d17ea32708c3-kube-api-access-sqcm8\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:15:08.479422 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.479373 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fdf2319-7997-41ea-832b-d17ea32708c3-extensions-socket-volume\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:15:08.548123 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.548032 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:08.579728 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.579691 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89252b48-4768-494e-826d-7f9ddd24bb0b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gbmgd\" (UID: \"89252b48-4768-494e-826d-7f9ddd24bb0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd" Sep 09 21:15:08.579895 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.579758 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvvjj\" (UniqueName: \"kubernetes.io/projected/89252b48-4768-494e-826d-7f9ddd24bb0b-kube-api-access-jvvjj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gbmgd\" (UID: \"89252b48-4768-494e-826d-7f9ddd24bb0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd" Sep 09 21:15:08.580115 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.580093 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89252b48-4768-494e-826d-7f9ddd24bb0b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gbmgd\" (UID: \"89252b48-4768-494e-826d-7f9ddd24bb0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd" Sep 09 21:15:08.590635 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.590609 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvvjj\" (UniqueName: \"kubernetes.io/projected/89252b48-4768-494e-826d-7f9ddd24bb0b-kube-api-access-jvvjj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gbmgd\" (UID: \"89252b48-4768-494e-826d-7f9ddd24bb0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd" Sep 09 21:15:08.705830 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.705789 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:08.711590 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:08.711561 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd" Sep 09 21:15:09.009906 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.009877 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd"] Sep 09 21:15:09.010438 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:15:09.010414 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89252b48_4768_494e_826d_7f9ddd24bb0b.slice/crio-c28ed406e38c1de9a0db66e8509df2e43e71a65983b302515c9c121b0588004f WatchSource:0}: Error finding container c28ed406e38c1de9a0db66e8509df2e43e71a65983b302515c9c121b0588004f: Status 404 returned error can't find the container with id c28ed406e38c1de9a0db66e8509df2e43e71a65983b302515c9c121b0588004f Sep 09 21:15:09.034022 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.033993 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd" event={"ID":"89252b48-4768-494e-826d-7f9ddd24bb0b","Type":"ContainerStarted","Data":"c28ed406e38c1de9a0db66e8509df2e43e71a65983b302515c9c121b0588004f"} Sep 09 21:15:09.035352 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.035328 2560 generic.go:356] "Generic (PLEG): container finished" podID="8fdf2319-7997-41ea-832b-d17ea32708c3" containerID="177faf00a88ea53a2882289c92d987bcba87a764c9de6ca0625e134c3947f8c5" exitCode=0 Sep 09 21:15:09.035435 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.035371 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" Sep 09 21:15:09.035475 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.035369 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" event={"ID":"8fdf2319-7997-41ea-832b-d17ea32708c3","Type":"ContainerDied","Data":"177faf00a88ea53a2882289c92d987bcba87a764c9de6ca0625e134c3947f8c5"} Sep 09 21:15:09.035510 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.035489 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd" event={"ID":"8fdf2319-7997-41ea-832b-d17ea32708c3","Type":"ContainerDied","Data":"13a7dc8c95ec8c70680d805a427bc1d4e04ea65460a1c32d21064fbc95114802"} Sep 09 21:15:09.035541 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.035517 2560 scope.go:117] "RemoveContainer" containerID="177faf00a88ea53a2882289c92d987bcba87a764c9de6ca0625e134c3947f8c5" Sep 09 21:15:09.063129 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.057873 2560 scope.go:117] "RemoveContainer" containerID="177faf00a88ea53a2882289c92d987bcba87a764c9de6ca0625e134c3947f8c5" Sep 09 21:15:09.063129 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:15:09.059530 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"177faf00a88ea53a2882289c92d987bcba87a764c9de6ca0625e134c3947f8c5\": container with ID starting with 177faf00a88ea53a2882289c92d987bcba87a764c9de6ca0625e134c3947f8c5 not found: ID does not exist" containerID="177faf00a88ea53a2882289c92d987bcba87a764c9de6ca0625e134c3947f8c5" Sep 09 21:15:09.063129 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.059576 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"177faf00a88ea53a2882289c92d987bcba87a764c9de6ca0625e134c3947f8c5"} err="failed to get container status \"177faf00a88ea53a2882289c92d987bcba87a764c9de6ca0625e134c3947f8c5\": rpc error: code = NotFound desc = could not find container \"177faf00a88ea53a2882289c92d987bcba87a764c9de6ca0625e134c3947f8c5\": container with ID starting with 177faf00a88ea53a2882289c92d987bcba87a764c9de6ca0625e134c3947f8c5 not found: ID does not exist" Sep 09 21:15:09.075367 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.075342 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd"] Sep 09 21:15:09.082195 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.082176 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hgcsd"] Sep 09 21:15:09.341034 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.340937 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:09.449574 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.449530 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:09.576499 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.576458 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:09.738899 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:09.738863 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:10.043263 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:10.043159 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd" event={"ID":"89252b48-4768-494e-826d-7f9ddd24bb0b","Type":"ContainerStarted","Data":"0199ee22b96fa1434c5e2ff6c22a98daf4db6d99622de5961fd2b7dd16bec19d"} Sep 09 21:15:10.043686 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:10.043376 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd" Sep 09 21:15:10.076640 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:10.076583 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd" podStartSLOduration=2.076566774 podStartE2EDuration="2.076566774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:15:10.07545991 +0000 UTC m=+825.936982307" watchObservedRunningTime="2026-09-09 21:15:10.076566774 +0000 UTC m=+825.938089168" Sep 09 21:15:10.779480 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:10.779447 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fdf2319-7997-41ea-832b-d17ea32708c3" path="/var/lib/kubelet/pods/8fdf2319-7997-41ea-832b-d17ea32708c3/volumes" Sep 09 21:15:16.788102 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:16.788058 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:16.883139 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:16.883106 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:21.051829 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:21.051796 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gbmgd" Sep 09 21:15:21.125000 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:21.124966 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8"] Sep 09 21:15:21.125227 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:21.125202 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" podUID="897abd07-1df0-4e7f-aed0-be88b430bbf1" containerName="manager" containerID="cri-o://ef311284a960674b5eb2cd582f3b67100aab84bdda95b54dd9521a3548907647" gracePeriod=10 Sep 09 21:15:21.373541 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:21.373515 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" Sep 09 21:15:21.487703 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:21.487667 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfrmq\" (UniqueName: \"kubernetes.io/projected/897abd07-1df0-4e7f-aed0-be88b430bbf1-kube-api-access-tfrmq\") pod \"897abd07-1df0-4e7f-aed0-be88b430bbf1\" (UID: \"897abd07-1df0-4e7f-aed0-be88b430bbf1\") " Sep 09 21:15:21.487904 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:21.487734 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/897abd07-1df0-4e7f-aed0-be88b430bbf1-extensions-socket-volume\") pod \"897abd07-1df0-4e7f-aed0-be88b430bbf1\" (UID: \"897abd07-1df0-4e7f-aed0-be88b430bbf1\") " Sep 09 21:15:21.488151 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:21.488126 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/897abd07-1df0-4e7f-aed0-be88b430bbf1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "897abd07-1df0-4e7f-aed0-be88b430bbf1" (UID: "897abd07-1df0-4e7f-aed0-be88b430bbf1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:15:21.489840 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:21.489809 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/897abd07-1df0-4e7f-aed0-be88b430bbf1-kube-api-access-tfrmq" (OuterVolumeSpecName: "kube-api-access-tfrmq") pod "897abd07-1df0-4e7f-aed0-be88b430bbf1" (UID: "897abd07-1df0-4e7f-aed0-be88b430bbf1"). InnerVolumeSpecName "kube-api-access-tfrmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:15:21.588509 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:21.588472 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfrmq\" (UniqueName: \"kubernetes.io/projected/897abd07-1df0-4e7f-aed0-be88b430bbf1-kube-api-access-tfrmq\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:15:21.588509 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:21.588503 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/897abd07-1df0-4e7f-aed0-be88b430bbf1-extensions-socket-volume\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:15:22.112140 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:22.112100 2560 generic.go:356] "Generic (PLEG): container finished" podID="897abd07-1df0-4e7f-aed0-be88b430bbf1" containerID="ef311284a960674b5eb2cd582f3b67100aab84bdda95b54dd9521a3548907647" exitCode=0 Sep 09 21:15:22.112580 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:22.112164 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" Sep 09 21:15:22.112580 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:22.112176 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" event={"ID":"897abd07-1df0-4e7f-aed0-be88b430bbf1","Type":"ContainerDied","Data":"ef311284a960674b5eb2cd582f3b67100aab84bdda95b54dd9521a3548907647"} Sep 09 21:15:22.112580 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:22.112257 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8" event={"ID":"897abd07-1df0-4e7f-aed0-be88b430bbf1","Type":"ContainerDied","Data":"ee3733dd5f6f9ebb6f0d3d9ea449085218c98941c2e4891ad492b3e8027111a0"} Sep 09 21:15:22.112580 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:22.112279 2560 scope.go:117] "RemoveContainer" containerID="ef311284a960674b5eb2cd582f3b67100aab84bdda95b54dd9521a3548907647" Sep 09 21:15:22.121625 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:22.121608 2560 scope.go:117] "RemoveContainer" containerID="ef311284a960674b5eb2cd582f3b67100aab84bdda95b54dd9521a3548907647" Sep 09 21:15:22.121898 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:15:22.121876 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef311284a960674b5eb2cd582f3b67100aab84bdda95b54dd9521a3548907647\": container with ID starting with ef311284a960674b5eb2cd582f3b67100aab84bdda95b54dd9521a3548907647 not found: ID does not exist" containerID="ef311284a960674b5eb2cd582f3b67100aab84bdda95b54dd9521a3548907647" Sep 09 21:15:22.121940 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:22.121905 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef311284a960674b5eb2cd582f3b67100aab84bdda95b54dd9521a3548907647"} err="failed to get container status \"ef311284a960674b5eb2cd582f3b67100aab84bdda95b54dd9521a3548907647\": rpc error: code = NotFound desc = could not find container \"ef311284a960674b5eb2cd582f3b67100aab84bdda95b54dd9521a3548907647\": container with ID starting with ef311284a960674b5eb2cd582f3b67100aab84bdda95b54dd9521a3548907647 not found: ID does not exist" Sep 09 21:15:22.146582 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:22.146553 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8"] Sep 09 21:15:22.157805 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:22.157779 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qnlx8"] Sep 09 21:15:22.779363 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:22.779332 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="897abd07-1df0-4e7f-aed0-be88b430bbf1" path="/var/lib/kubelet/pods/897abd07-1df0-4e7f-aed0-be88b430bbf1/volumes" Sep 09 21:15:24.675856 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:24.675816 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:25.480876 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:25.480843 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:39.000909 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.000870 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:39.007140 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.007115 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zvx9g"] Sep 09 21:15:39.007426 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.007415 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="897abd07-1df0-4e7f-aed0-be88b430bbf1" containerName="manager" Sep 09 21:15:39.007470 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.007428 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="897abd07-1df0-4e7f-aed0-be88b430bbf1" containerName="manager" Sep 09 21:15:39.007506 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.007487 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="897abd07-1df0-4e7f-aed0-be88b430bbf1" containerName="manager" Sep 09 21:15:39.010311 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.010295 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" Sep 09 21:15:39.013895 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.013873 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 09 21:15:39.023735 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.023711 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zvx9g"] Sep 09 21:15:39.096983 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.096951 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zvx9g"] Sep 09 21:15:39.131903 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.131868 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9wqz\" (UniqueName: \"kubernetes.io/projected/0a928f91-6514-4b74-ae18-4a0b3544fd26-kube-api-access-w9wqz\") pod \"limitador-limitador-8588749977-zvx9g\" (UID: \"0a928f91-6514-4b74-ae18-4a0b3544fd26\") " pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" Sep 09 21:15:39.132083 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.131968 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0a928f91-6514-4b74-ae18-4a0b3544fd26-config-file\") pod \"limitador-limitador-8588749977-zvx9g\" (UID: \"0a928f91-6514-4b74-ae18-4a0b3544fd26\") " pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" Sep 09 21:15:39.193109 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.193069 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:39.232970 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.232930 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9wqz\" (UniqueName: \"kubernetes.io/projected/0a928f91-6514-4b74-ae18-4a0b3544fd26-kube-api-access-w9wqz\") pod \"limitador-limitador-8588749977-zvx9g\" (UID: \"0a928f91-6514-4b74-ae18-4a0b3544fd26\") " pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" Sep 09 21:15:39.233148 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.232988 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0a928f91-6514-4b74-ae18-4a0b3544fd26-config-file\") pod \"limitador-limitador-8588749977-zvx9g\" (UID: \"0a928f91-6514-4b74-ae18-4a0b3544fd26\") " pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" Sep 09 21:15:39.233594 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.233575 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0a928f91-6514-4b74-ae18-4a0b3544fd26-config-file\") pod \"limitador-limitador-8588749977-zvx9g\" (UID: \"0a928f91-6514-4b74-ae18-4a0b3544fd26\") " pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" Sep 09 21:15:39.245810 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.245775 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9wqz\" (UniqueName: \"kubernetes.io/projected/0a928f91-6514-4b74-ae18-4a0b3544fd26-kube-api-access-w9wqz\") pod \"limitador-limitador-8588749977-zvx9g\" (UID: \"0a928f91-6514-4b74-ae18-4a0b3544fd26\") " pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" Sep 09 21:15:39.289271 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.289183 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:39.320082 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.320028 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" Sep 09 21:15:39.595729 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.595701 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zvx9g"] Sep 09 21:15:39.597450 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:15:39.597424 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a928f91_6514_4b74_ae18_4a0b3544fd26.slice/crio-2295b756cdde397ce740fcfd52903f71d88a3d1c9be57996076a2366394f8de4 WatchSource:0}: Error finding container 2295b756cdde397ce740fcfd52903f71d88a3d1c9be57996076a2366394f8de4: Status 404 returned error can't find the container with id 2295b756cdde397ce740fcfd52903f71d88a3d1c9be57996076a2366394f8de4 Sep 09 21:15:39.915731 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:39.915626 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:40.022075 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:40.021998 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:40.119821 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:40.119774 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-nn6hz"] Sep 09 21:15:40.122751 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:40.122729 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-nn6hz" Sep 09 21:15:40.125269 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:40.125251 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nm6dw\"" Sep 09 21:15:40.132977 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:40.132952 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-nn6hz"] Sep 09 21:15:40.211644 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:40.211611 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" event={"ID":"0a928f91-6514-4b74-ae18-4a0b3544fd26","Type":"ContainerStarted","Data":"2295b756cdde397ce740fcfd52903f71d88a3d1c9be57996076a2366394f8de4"} Sep 09 21:15:40.242178 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:40.242140 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9kkn\" (UniqueName: \"kubernetes.io/projected/46415bf2-6a4a-4c2c-8372-7249c1542e0b-kube-api-access-t9kkn\") pod \"authorino-59c974bd48-nn6hz\" (UID: \"46415bf2-6a4a-4c2c-8372-7249c1542e0b\") " pod="kuadrant-system/authorino-59c974bd48-nn6hz" Sep 09 21:15:40.343149 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:40.343106 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9kkn\" (UniqueName: \"kubernetes.io/projected/46415bf2-6a4a-4c2c-8372-7249c1542e0b-kube-api-access-t9kkn\") pod \"authorino-59c974bd48-nn6hz\" (UID: \"46415bf2-6a4a-4c2c-8372-7249c1542e0b\") " pod="kuadrant-system/authorino-59c974bd48-nn6hz" Sep 09 21:15:40.356075 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:40.356030 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9kkn\" (UniqueName: \"kubernetes.io/projected/46415bf2-6a4a-4c2c-8372-7249c1542e0b-kube-api-access-t9kkn\") pod \"authorino-59c974bd48-nn6hz\" (UID: \"46415bf2-6a4a-4c2c-8372-7249c1542e0b\") " pod="kuadrant-system/authorino-59c974bd48-nn6hz" Sep 09 21:15:40.432485 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:40.432449 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-nn6hz" Sep 09 21:15:40.710082 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:40.710053 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-nn6hz"] Sep 09 21:15:40.711671 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:15:40.711644 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46415bf2_6a4a_4c2c_8372_7249c1542e0b.slice/crio-532e33c17cdeb55103e253cd61f2577226fbb1c2580e907b9d23a990974aa14a WatchSource:0}: Error finding container 532e33c17cdeb55103e253cd61f2577226fbb1c2580e907b9d23a990974aa14a: Status 404 returned error can't find the container with id 532e33c17cdeb55103e253cd61f2577226fbb1c2580e907b9d23a990974aa14a Sep 09 21:15:41.220789 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:41.220739 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-nn6hz" event={"ID":"46415bf2-6a4a-4c2c-8372-7249c1542e0b","Type":"ContainerStarted","Data":"532e33c17cdeb55103e253cd61f2577226fbb1c2580e907b9d23a990974aa14a"} Sep 09 21:15:42.229234 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:42.229185 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" event={"ID":"0a928f91-6514-4b74-ae18-4a0b3544fd26","Type":"ContainerStarted","Data":"9f8f62fa30cf8332ae5df7f668747fac6a8a18b891d1dc49b4157359a516e869"} Sep 09 21:15:42.229729 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:42.229388 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" Sep 09 21:15:42.250766 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:42.250721 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" podStartSLOduration=2.558242414 podStartE2EDuration="4.250702204s" totalImagesPullingTime="1.69245979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:15:42.248397536 +0000 UTC m=+858.109919932" watchObservedRunningTime="2026-09-09 21:15:42.250702204 +0000 UTC m=+858.112224603" Sep 09 21:15:43.235429 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:43.235392 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-nn6hz" event={"ID":"46415bf2-6a4a-4c2c-8372-7249c1542e0b","Type":"ContainerStarted","Data":"455015c4f4cbe634f2775cd0e796c1bbe27bd5ef872e246cd746b300c40dbcad"} Sep 09 21:15:53.236359 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:53.236325 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" Sep 09 21:15:53.258347 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:53.258295 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-nn6hz" podStartSLOduration=11.548817796 podStartE2EDuration="13.258277469s" totalImagesPullingTime="1.709459673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:15:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:15:43.25554693 +0000 UTC m=+859.117069325" watchObservedRunningTime="2026-09-09 21:15:53.258277469 +0000 UTC m=+869.119799864" Sep 09 21:15:54.508685 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:54.508646 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zvx9g"] Sep 09 21:15:54.509582 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:54.508894 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" podUID="0a928f91-6514-4b74-ae18-4a0b3544fd26" containerName="limitador" containerID="cri-o://9f8f62fa30cf8332ae5df7f668747fac6a8a18b891d1dc49b4157359a516e869" gracePeriod=30 Sep 09 21:15:55.065852 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.065822 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" Sep 09 21:15:55.165115 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.164993 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0a928f91-6514-4b74-ae18-4a0b3544fd26-config-file\") pod \"0a928f91-6514-4b74-ae18-4a0b3544fd26\" (UID: \"0a928f91-6514-4b74-ae18-4a0b3544fd26\") " Sep 09 21:15:55.165115 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.165101 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9wqz\" (UniqueName: \"kubernetes.io/projected/0a928f91-6514-4b74-ae18-4a0b3544fd26-kube-api-access-w9wqz\") pod \"0a928f91-6514-4b74-ae18-4a0b3544fd26\" (UID: \"0a928f91-6514-4b74-ae18-4a0b3544fd26\") " Sep 09 21:15:55.165413 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.165393 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a928f91-6514-4b74-ae18-4a0b3544fd26-config-file" (OuterVolumeSpecName: "config-file") pod "0a928f91-6514-4b74-ae18-4a0b3544fd26" (UID: "0a928f91-6514-4b74-ae18-4a0b3544fd26"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:15:55.167114 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.167087 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a928f91-6514-4b74-ae18-4a0b3544fd26-kube-api-access-w9wqz" (OuterVolumeSpecName: "kube-api-access-w9wqz") pod "0a928f91-6514-4b74-ae18-4a0b3544fd26" (UID: "0a928f91-6514-4b74-ae18-4a0b3544fd26"). InnerVolumeSpecName "kube-api-access-w9wqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:15:55.266090 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.266053 2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0a928f91-6514-4b74-ae18-4a0b3544fd26-config-file\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:15:55.266090 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.266083 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9wqz\" (UniqueName: \"kubernetes.io/projected/0a928f91-6514-4b74-ae18-4a0b3544fd26-kube-api-access-w9wqz\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:15:55.296893 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.296859 2560 generic.go:356] "Generic (PLEG): container finished" podID="0a928f91-6514-4b74-ae18-4a0b3544fd26" containerID="9f8f62fa30cf8332ae5df7f668747fac6a8a18b891d1dc49b4157359a516e869" exitCode=0 Sep 09 21:15:55.297079 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.296909 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" Sep 09 21:15:55.297079 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.296947 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" event={"ID":"0a928f91-6514-4b74-ae18-4a0b3544fd26","Type":"ContainerDied","Data":"9f8f62fa30cf8332ae5df7f668747fac6a8a18b891d1dc49b4157359a516e869"} Sep 09 21:15:55.297079 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.297007 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-zvx9g" event={"ID":"0a928f91-6514-4b74-ae18-4a0b3544fd26","Type":"ContainerDied","Data":"2295b756cdde397ce740fcfd52903f71d88a3d1c9be57996076a2366394f8de4"} Sep 09 21:15:55.297079 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.297037 2560 scope.go:117] "RemoveContainer" containerID="9f8f62fa30cf8332ae5df7f668747fac6a8a18b891d1dc49b4157359a516e869" Sep 09 21:15:55.305993 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.305977 2560 scope.go:117] "RemoveContainer" containerID="9f8f62fa30cf8332ae5df7f668747fac6a8a18b891d1dc49b4157359a516e869" Sep 09 21:15:55.306289 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:15:55.306270 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f8f62fa30cf8332ae5df7f668747fac6a8a18b891d1dc49b4157359a516e869\": container with ID starting with 9f8f62fa30cf8332ae5df7f668747fac6a8a18b891d1dc49b4157359a516e869 not found: ID does not exist" containerID="9f8f62fa30cf8332ae5df7f668747fac6a8a18b891d1dc49b4157359a516e869" Sep 09 21:15:55.306346 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.306297 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f8f62fa30cf8332ae5df7f668747fac6a8a18b891d1dc49b4157359a516e869"} err="failed to get container status \"9f8f62fa30cf8332ae5df7f668747fac6a8a18b891d1dc49b4157359a516e869\": rpc error: code = NotFound desc = could not find container \"9f8f62fa30cf8332ae5df7f668747fac6a8a18b891d1dc49b4157359a516e869\": container with ID starting with 9f8f62fa30cf8332ae5df7f668747fac6a8a18b891d1dc49b4157359a516e869 not found: ID does not exist" Sep 09 21:15:55.320935 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.320911 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zvx9g"] Sep 09 21:15:55.329837 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:55.329816 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zvx9g"] Sep 09 21:15:56.779403 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:56.779372 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a928f91-6514-4b74-ae18-4a0b3544fd26" path="/var/lib/kubelet/pods/0a928f91-6514-4b74-ae18-4a0b3544fd26/volumes" Sep 09 21:15:58.580062 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:58.580013 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:58.665508 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:58.665476 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:59.070632 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.070594 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4"] Sep 09 21:15:59.070973 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.070960 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a928f91-6514-4b74-ae18-4a0b3544fd26" containerName="limitador" Sep 09 21:15:59.071019 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.070975 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a928f91-6514-4b74-ae18-4a0b3544fd26" containerName="limitador" Sep 09 21:15:59.071080 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.071053 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a928f91-6514-4b74-ae18-4a0b3544fd26" containerName="limitador" Sep 09 21:15:59.073529 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.073510 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4" Sep 09 21:15:59.076955 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.076930 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 09 21:15:59.077086 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.076979 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 09 21:15:59.077374 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.077355 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-t6tq6\"" Sep 09 21:15:59.077477 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.077399 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 09 21:15:59.085573 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.085544 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4"] Sep 09 21:15:59.094883 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.094861 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:15:59.098194 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.098175 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c8d843b5-e932-44fa-825a-46f1c81fd4c6-data\") pod \"postgres-69fb4594cf-zmhs4\" (UID: \"c8d843b5-e932-44fa-825a-46f1c81fd4c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4" Sep 09 21:15:59.098290 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.098223 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkjtb\" (UniqueName: \"kubernetes.io/projected/c8d843b5-e932-44fa-825a-46f1c81fd4c6-kube-api-access-jkjtb\") pod \"postgres-69fb4594cf-zmhs4\" (UID: \"c8d843b5-e932-44fa-825a-46f1c81fd4c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4" Sep 09 21:15:59.199547 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.199512 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkjtb\" (UniqueName: \"kubernetes.io/projected/c8d843b5-e932-44fa-825a-46f1c81fd4c6-kube-api-access-jkjtb\") pod \"postgres-69fb4594cf-zmhs4\" (UID: \"c8d843b5-e932-44fa-825a-46f1c81fd4c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4" Sep 09 21:15:59.199708 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.199582 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c8d843b5-e932-44fa-825a-46f1c81fd4c6-data\") pod \"postgres-69fb4594cf-zmhs4\" (UID: \"c8d843b5-e932-44fa-825a-46f1c81fd4c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4" Sep 09 21:15:59.199917 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.199900 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c8d843b5-e932-44fa-825a-46f1c81fd4c6-data\") pod \"postgres-69fb4594cf-zmhs4\" (UID: \"c8d843b5-e932-44fa-825a-46f1c81fd4c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4" Sep 09 21:15:59.211435 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.211407 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkjtb\" (UniqueName: \"kubernetes.io/projected/c8d843b5-e932-44fa-825a-46f1c81fd4c6-kube-api-access-jkjtb\") pod \"postgres-69fb4594cf-zmhs4\" (UID: \"c8d843b5-e932-44fa-825a-46f1c81fd4c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4" Sep 09 21:15:59.383491 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.383396 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4" Sep 09 21:15:59.675838 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:15:59.675664 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4"] Sep 09 21:15:59.678205 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:15:59.678172 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8d843b5_e932_44fa_825a_46f1c81fd4c6.slice/crio-0486c9ab4681232d4075bdc3682f97868c247b26b4b3ba753d7ba7e9fa4316f2 WatchSource:0}: Error finding container 0486c9ab4681232d4075bdc3682f97868c247b26b4b3ba753d7ba7e9fa4316f2: Status 404 returned error can't find the container with id 0486c9ab4681232d4075bdc3682f97868c247b26b4b3ba753d7ba7e9fa4316f2 Sep 09 21:16:00.327413 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:00.327376 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4" event={"ID":"c8d843b5-e932-44fa-825a-46f1c81fd4c6","Type":"ContainerStarted","Data":"0486c9ab4681232d4075bdc3682f97868c247b26b4b3ba753d7ba7e9fa4316f2"} Sep 09 21:16:02.340626 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:02.340590 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4" event={"ID":"c8d843b5-e932-44fa-825a-46f1c81fd4c6","Type":"ContainerStarted","Data":"2b43dfe6e96f798cc9ba4fc7173ddb0cbed899b5f94140728ff4e2db919073b7"} Sep 09 21:16:02.341021 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:02.340791 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4" Sep 09 21:16:02.360753 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:02.360703 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4" podStartSLOduration=1.5063944120000001 podStartE2EDuration="3.360683358s" totalImagesPullingTime="1.854288946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:15:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:16:02.360518147 +0000 UTC m=+878.222040543" watchObservedRunningTime="2026-09-09 21:16:02.360683358 +0000 UTC m=+878.222205755" Sep 09 21:16:08.375501 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:08.375469 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zmhs4" Sep 09 21:16:08.719525 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:08.719487 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:08.950552 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:08.950518 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-nrv7z"] Sep 09 21:16:08.953469 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:08.953446 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nrv7z" Sep 09 21:16:08.966546 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:08.966517 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-nrv7z"] Sep 09 21:16:09.020433 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.020346 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:09.080640 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.080601 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5skg9\" (UniqueName: \"kubernetes.io/projected/2ce8f661-0e6b-473d-8967-7b1933d526f7-kube-api-access-5skg9\") pod \"authorino-557f876f69-nrv7z\" (UID: \"2ce8f661-0e6b-473d-8967-7b1933d526f7\") " pod="kuadrant-system/authorino-557f876f69-nrv7z" Sep 09 21:16:09.094646 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.094617 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-nrv7z"] Sep 09 21:16:09.094856 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:16:09.094837 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-5skg9], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-nrv7z" podUID="2ce8f661-0e6b-473d-8967-7b1933d526f7" Sep 09 21:16:09.158701 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.158661 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-dhhlh"] Sep 09 21:16:09.161058 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.161027 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-dhhlh" Sep 09 21:16:09.166990 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.166965 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 09 21:16:09.176962 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.176940 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-dhhlh"] Sep 09 21:16:09.181872 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.181848 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5skg9\" (UniqueName: \"kubernetes.io/projected/2ce8f661-0e6b-473d-8967-7b1933d526f7-kube-api-access-5skg9\") pod \"authorino-557f876f69-nrv7z\" (UID: \"2ce8f661-0e6b-473d-8967-7b1933d526f7\") " pod="kuadrant-system/authorino-557f876f69-nrv7z" Sep 09 21:16:09.193610 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.193581 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5skg9\" (UniqueName: \"kubernetes.io/projected/2ce8f661-0e6b-473d-8967-7b1933d526f7-kube-api-access-5skg9\") pod \"authorino-557f876f69-nrv7z\" (UID: \"2ce8f661-0e6b-473d-8967-7b1933d526f7\") " pod="kuadrant-system/authorino-557f876f69-nrv7z" Sep 09 21:16:09.283145 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.283036 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj2q4\" (UniqueName: \"kubernetes.io/projected/8ba51261-b389-4a8f-8e84-bf3b902c0137-kube-api-access-nj2q4\") pod \"authorino-6d4dfc649f-dhhlh\" (UID: \"8ba51261-b389-4a8f-8e84-bf3b902c0137\") " pod="kuadrant-system/authorino-6d4dfc649f-dhhlh" Sep 09 21:16:09.283145 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.283111 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ba51261-b389-4a8f-8e84-bf3b902c0137-tls-cert\") pod \"authorino-6d4dfc649f-dhhlh\" (UID: \"8ba51261-b389-4a8f-8e84-bf3b902c0137\") " pod="kuadrant-system/authorino-6d4dfc649f-dhhlh" Sep 09 21:16:09.324168 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.324129 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-dhhlh"] Sep 09 21:16:09.324387 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:16:09.324366 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-nj2q4 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-dhhlh" podUID="8ba51261-b389-4a8f-8e84-bf3b902c0137" Sep 09 21:16:09.376508 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.376474 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nrv7z" Sep 09 21:16:09.376920 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.376474 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-dhhlh" Sep 09 21:16:09.382577 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.382555 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nrv7z" Sep 09 21:16:09.383968 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.383945 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj2q4\" (UniqueName: \"kubernetes.io/projected/8ba51261-b389-4a8f-8e84-bf3b902c0137-kube-api-access-nj2q4\") pod \"authorino-6d4dfc649f-dhhlh\" (UID: \"8ba51261-b389-4a8f-8e84-bf3b902c0137\") " pod="kuadrant-system/authorino-6d4dfc649f-dhhlh" Sep 09 21:16:09.384127 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.383985 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ba51261-b389-4a8f-8e84-bf3b902c0137-tls-cert\") pod \"authorino-6d4dfc649f-dhhlh\" (UID: \"8ba51261-b389-4a8f-8e84-bf3b902c0137\") " pod="kuadrant-system/authorino-6d4dfc649f-dhhlh" Sep 09 21:16:09.385667 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.385642 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-dhhlh" Sep 09 21:16:09.386482 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.386463 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ba51261-b389-4a8f-8e84-bf3b902c0137-tls-cert\") pod \"authorino-6d4dfc649f-dhhlh\" (UID: \"8ba51261-b389-4a8f-8e84-bf3b902c0137\") " pod="kuadrant-system/authorino-6d4dfc649f-dhhlh" Sep 09 21:16:09.405968 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.405945 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj2q4\" (UniqueName: \"kubernetes.io/projected/8ba51261-b389-4a8f-8e84-bf3b902c0137-kube-api-access-nj2q4\") pod \"authorino-6d4dfc649f-dhhlh\" (UID: \"8ba51261-b389-4a8f-8e84-bf3b902c0137\") " pod="kuadrant-system/authorino-6d4dfc649f-dhhlh" Sep 09 21:16:09.439680 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.439654 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84cbdc795c-dmh52"] Sep 09 21:16:09.441861 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.441840 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84cbdc795c-dmh52" Sep 09 21:16:09.463743 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.463721 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84cbdc795c-dmh52"] Sep 09 21:16:09.484566 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.484541 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj2q4\" (UniqueName: \"kubernetes.io/projected/8ba51261-b389-4a8f-8e84-bf3b902c0137-kube-api-access-nj2q4\") pod \"8ba51261-b389-4a8f-8e84-bf3b902c0137\" (UID: \"8ba51261-b389-4a8f-8e84-bf3b902c0137\") " Sep 09 21:16:09.484698 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.484582 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ba51261-b389-4a8f-8e84-bf3b902c0137-tls-cert\") pod \"8ba51261-b389-4a8f-8e84-bf3b902c0137\" (UID: \"8ba51261-b389-4a8f-8e84-bf3b902c0137\") " Sep 09 21:16:09.484698 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.484601 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5skg9\" (UniqueName: \"kubernetes.io/projected/2ce8f661-0e6b-473d-8967-7b1933d526f7-kube-api-access-5skg9\") pod \"2ce8f661-0e6b-473d-8967-7b1933d526f7\" (UID: \"2ce8f661-0e6b-473d-8967-7b1933d526f7\") " Sep 09 21:16:09.484824 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.484703 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmhft\" (UniqueName: \"kubernetes.io/projected/58226d15-493f-4037-8628-e82e51578987-kube-api-access-xmhft\") pod \"authorino-84cbdc795c-dmh52\" (UID: \"58226d15-493f-4037-8628-e82e51578987\") " pod="kuadrant-system/authorino-84cbdc795c-dmh52" Sep 09 21:16:09.484824 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.484752 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58226d15-493f-4037-8628-e82e51578987-tls-cert\") pod \"authorino-84cbdc795c-dmh52\" (UID: \"58226d15-493f-4037-8628-e82e51578987\") " pod="kuadrant-system/authorino-84cbdc795c-dmh52" Sep 09 21:16:09.486795 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.486765 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ba51261-b389-4a8f-8e84-bf3b902c0137-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "8ba51261-b389-4a8f-8e84-bf3b902c0137" (UID: "8ba51261-b389-4a8f-8e84-bf3b902c0137"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:16:09.486795 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.486768 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ba51261-b389-4a8f-8e84-bf3b902c0137-kube-api-access-nj2q4" (OuterVolumeSpecName: "kube-api-access-nj2q4") pod "8ba51261-b389-4a8f-8e84-bf3b902c0137" (UID: "8ba51261-b389-4a8f-8e84-bf3b902c0137"). InnerVolumeSpecName "kube-api-access-nj2q4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:16:09.486924 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.486810 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ce8f661-0e6b-473d-8967-7b1933d526f7-kube-api-access-5skg9" (OuterVolumeSpecName: "kube-api-access-5skg9") pod "2ce8f661-0e6b-473d-8967-7b1933d526f7" (UID: "2ce8f661-0e6b-473d-8967-7b1933d526f7"). InnerVolumeSpecName "kube-api-access-5skg9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:16:09.585887 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.585799 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmhft\" (UniqueName: \"kubernetes.io/projected/58226d15-493f-4037-8628-e82e51578987-kube-api-access-xmhft\") pod \"authorino-84cbdc795c-dmh52\" (UID: \"58226d15-493f-4037-8628-e82e51578987\") " pod="kuadrant-system/authorino-84cbdc795c-dmh52" Sep 09 21:16:09.586081 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.585897 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58226d15-493f-4037-8628-e82e51578987-tls-cert\") pod \"authorino-84cbdc795c-dmh52\" (UID: \"58226d15-493f-4037-8628-e82e51578987\") " pod="kuadrant-system/authorino-84cbdc795c-dmh52" Sep 09 21:16:09.586081 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.585964 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5skg9\" (UniqueName: \"kubernetes.io/projected/2ce8f661-0e6b-473d-8967-7b1933d526f7-kube-api-access-5skg9\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:16:09.586081 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.585978 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj2q4\" (UniqueName: \"kubernetes.io/projected/8ba51261-b389-4a8f-8e84-bf3b902c0137-kube-api-access-nj2q4\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:16:09.586081 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.585991 2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ba51261-b389-4a8f-8e84-bf3b902c0137-tls-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:16:09.588687 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.588661 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58226d15-493f-4037-8628-e82e51578987-tls-cert\") pod \"authorino-84cbdc795c-dmh52\" (UID: \"58226d15-493f-4037-8628-e82e51578987\") " pod="kuadrant-system/authorino-84cbdc795c-dmh52" Sep 09 21:16:09.600916 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.600892 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmhft\" (UniqueName: \"kubernetes.io/projected/58226d15-493f-4037-8628-e82e51578987-kube-api-access-xmhft\") pod \"authorino-84cbdc795c-dmh52\" (UID: \"58226d15-493f-4037-8628-e82e51578987\") " pod="kuadrant-system/authorino-84cbdc795c-dmh52" Sep 09 21:16:09.750757 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:09.750724 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84cbdc795c-dmh52" Sep 09 21:16:10.035897 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:10.035864 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84cbdc795c-dmh52"] Sep 09 21:16:10.038684 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:16:10.038652 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58226d15_493f_4037_8628_e82e51578987.slice/crio-135010a8030cf977f5e0bed51c6a57e7929e93ed1a1a4783354b9e234a0fb3cd WatchSource:0}: Error finding container 135010a8030cf977f5e0bed51c6a57e7929e93ed1a1a4783354b9e234a0fb3cd: Status 404 returned error can't find the container with id 135010a8030cf977f5e0bed51c6a57e7929e93ed1a1a4783354b9e234a0fb3cd Sep 09 21:16:10.383430 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:10.383336 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84cbdc795c-dmh52" event={"ID":"58226d15-493f-4037-8628-e82e51578987","Type":"ContainerStarted","Data":"135010a8030cf977f5e0bed51c6a57e7929e93ed1a1a4783354b9e234a0fb3cd"} Sep 09 21:16:10.383430 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:10.383361 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nrv7z" Sep 09 21:16:10.383430 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:10.383348 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-dhhlh" Sep 09 21:16:10.431232 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:10.431201 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-dhhlh"] Sep 09 21:16:10.438384 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:10.438360 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-dhhlh"] Sep 09 21:16:10.465566 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:10.465537 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-nrv7z"] Sep 09 21:16:10.470268 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:10.470237 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-nrv7z"] Sep 09 21:16:10.779869 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:10.779836 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ce8f661-0e6b-473d-8967-7b1933d526f7" path="/var/lib/kubelet/pods/2ce8f661-0e6b-473d-8967-7b1933d526f7/volumes" Sep 09 21:16:10.780119 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:10.780106 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ba51261-b389-4a8f-8e84-bf3b902c0137" path="/var/lib/kubelet/pods/8ba51261-b389-4a8f-8e84-bf3b902c0137/volumes" Sep 09 21:16:11.390179 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:11.390145 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84cbdc795c-dmh52" event={"ID":"58226d15-493f-4037-8628-e82e51578987","Type":"ContainerStarted","Data":"ae8ea90e2e3db0fa086bb89bd6cf47f482ddc0683daf501353e68b96570e2ebc"} Sep 09 21:16:11.411515 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:11.411463 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-84cbdc795c-dmh52" podStartSLOduration=1.652319903 podStartE2EDuration="2.411447182s" totalImagesPullingTime="759.127279ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:16:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:16:11.409672795 +0000 UTC m=+887.271195190" watchObservedRunningTime="2026-09-09 21:16:11.411447182 +0000 UTC m=+887.272969577" Sep 09 21:16:11.443572 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:11.443538 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-nn6hz"] Sep 09 21:16:11.443774 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:11.443750 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-nn6hz" podUID="46415bf2-6a4a-4c2c-8372-7249c1542e0b" containerName="authorino" containerID="cri-o://455015c4f4cbe634f2775cd0e796c1bbe27bd5ef872e246cd746b300c40dbcad" gracePeriod=30 Sep 09 21:16:11.673656 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:11.673634 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-nn6hz" Sep 09 21:16:11.807314 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:11.807281 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9kkn\" (UniqueName: \"kubernetes.io/projected/46415bf2-6a4a-4c2c-8372-7249c1542e0b-kube-api-access-t9kkn\") pod \"46415bf2-6a4a-4c2c-8372-7249c1542e0b\" (UID: \"46415bf2-6a4a-4c2c-8372-7249c1542e0b\") " Sep 09 21:16:11.809280 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:11.809251 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46415bf2-6a4a-4c2c-8372-7249c1542e0b-kube-api-access-t9kkn" (OuterVolumeSpecName: "kube-api-access-t9kkn") pod "46415bf2-6a4a-4c2c-8372-7249c1542e0b" (UID: "46415bf2-6a4a-4c2c-8372-7249c1542e0b"). InnerVolumeSpecName "kube-api-access-t9kkn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:16:11.908372 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:11.908283 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9kkn\" (UniqueName: \"kubernetes.io/projected/46415bf2-6a4a-4c2c-8372-7249c1542e0b-kube-api-access-t9kkn\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:16:12.396677 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:12.396643 2560 generic.go:356] "Generic (PLEG): container finished" podID="46415bf2-6a4a-4c2c-8372-7249c1542e0b" containerID="455015c4f4cbe634f2775cd0e796c1bbe27bd5ef872e246cd746b300c40dbcad" exitCode=0 Sep 09 21:16:12.397168 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:12.396683 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-nn6hz" Sep 09 21:16:12.397168 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:12.396723 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-nn6hz" event={"ID":"46415bf2-6a4a-4c2c-8372-7249c1542e0b","Type":"ContainerDied","Data":"455015c4f4cbe634f2775cd0e796c1bbe27bd5ef872e246cd746b300c40dbcad"} Sep 09 21:16:12.397168 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:12.396764 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-nn6hz" event={"ID":"46415bf2-6a4a-4c2c-8372-7249c1542e0b","Type":"ContainerDied","Data":"532e33c17cdeb55103e253cd61f2577226fbb1c2580e907b9d23a990974aa14a"} Sep 09 21:16:12.397168 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:12.396791 2560 scope.go:117] "RemoveContainer" containerID="455015c4f4cbe634f2775cd0e796c1bbe27bd5ef872e246cd746b300c40dbcad" Sep 09 21:16:12.407757 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:12.407726 2560 scope.go:117] "RemoveContainer" containerID="455015c4f4cbe634f2775cd0e796c1bbe27bd5ef872e246cd746b300c40dbcad" Sep 09 21:16:12.408059 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:16:12.408027 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"455015c4f4cbe634f2775cd0e796c1bbe27bd5ef872e246cd746b300c40dbcad\": container with ID starting with 455015c4f4cbe634f2775cd0e796c1bbe27bd5ef872e246cd746b300c40dbcad not found: ID does not exist" containerID="455015c4f4cbe634f2775cd0e796c1bbe27bd5ef872e246cd746b300c40dbcad" Sep 09 21:16:12.408119 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:12.408071 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"455015c4f4cbe634f2775cd0e796c1bbe27bd5ef872e246cd746b300c40dbcad"} err="failed to get container status \"455015c4f4cbe634f2775cd0e796c1bbe27bd5ef872e246cd746b300c40dbcad\": rpc error: code = NotFound desc = could not find container \"455015c4f4cbe634f2775cd0e796c1bbe27bd5ef872e246cd746b300c40dbcad\": container with ID starting with 455015c4f4cbe634f2775cd0e796c1bbe27bd5ef872e246cd746b300c40dbcad not found: ID does not exist" Sep 09 21:16:12.442509 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:12.442441 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-nn6hz"] Sep 09 21:16:12.444499 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:12.444477 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-nn6hz"] Sep 09 21:16:12.784398 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:12.784365 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46415bf2-6a4a-4c2c-8372-7249c1542e0b" path="/var/lib/kubelet/pods/46415bf2-6a4a-4c2c-8372-7249c1542e0b/volumes" Sep 09 21:16:16.483861 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.483826 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:16.555366 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.555334 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6886dd5f96-w5rhj"] Sep 09 21:16:16.555802 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.555778 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46415bf2-6a4a-4c2c-8372-7249c1542e0b" containerName="authorino" Sep 09 21:16:16.555947 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.555818 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="46415bf2-6a4a-4c2c-8372-7249c1542e0b" containerName="authorino" Sep 09 21:16:16.556013 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.555952 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="46415bf2-6a4a-4c2c-8372-7249c1542e0b" containerName="authorino" Sep 09 21:16:16.563924 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.563900 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:16.568387 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.568364 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 09 21:16:16.568619 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.568417 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-s9fjv\"" Sep 09 21:16:16.568726 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.568504 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 09 21:16:16.569496 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.569478 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 09 21:16:16.576298 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.576279 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6886dd5f96-w5rhj"] Sep 09 21:16:16.586273 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.586242 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:16.652513 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.652479 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf2m7\" (UniqueName: \"kubernetes.io/projected/f5b731af-c9bf-48c8-98dd-ad6becb06867-kube-api-access-qf2m7\") pod \"maas-controller-6886dd5f96-w5rhj\" (UID: \"f5b731af-c9bf-48c8-98dd-ad6becb06867\") " pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:16.652762 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.652521 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-metrics-certs\") pod \"maas-controller-6886dd5f96-w5rhj\" (UID: \"f5b731af-c9bf-48c8-98dd-ad6becb06867\") " pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:16.652762 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.652556 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-webhook-cert\") pod \"maas-controller-6886dd5f96-w5rhj\" (UID: \"f5b731af-c9bf-48c8-98dd-ad6becb06867\") " pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:16.660006 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.659966 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:16.753268 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.753180 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-webhook-cert\") pod \"maas-controller-6886dd5f96-w5rhj\" (UID: \"f5b731af-c9bf-48c8-98dd-ad6becb06867\") " pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:16.753268 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.753261 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf2m7\" (UniqueName: \"kubernetes.io/projected/f5b731af-c9bf-48c8-98dd-ad6becb06867-kube-api-access-qf2m7\") pod \"maas-controller-6886dd5f96-w5rhj\" (UID: \"f5b731af-c9bf-48c8-98dd-ad6becb06867\") " pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:16.753464 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.753303 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-metrics-certs\") pod \"maas-controller-6886dd5f96-w5rhj\" (UID: \"f5b731af-c9bf-48c8-98dd-ad6becb06867\") " pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:16.753464 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:16:16.753345 2560 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 09 21:16:16.753464 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:16:16.753427 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-webhook-cert podName:f5b731af-c9bf-48c8-98dd-ad6becb06867 nodeName:}" failed. No retries permitted until 2026-09-09 21:16:17.253403163 +0000 UTC m=+893.114925537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-webhook-cert") pod "maas-controller-6886dd5f96-w5rhj" (UID: "f5b731af-c9bf-48c8-98dd-ad6becb06867") : secret "maas-controller-webhook-cert" not found Sep 09 21:16:16.753464 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:16:16.753430 2560 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 09 21:16:16.753634 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:16:16.753493 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-metrics-certs podName:f5b731af-c9bf-48c8-98dd-ad6becb06867 nodeName:}" failed. No retries permitted until 2026-09-09 21:16:17.253475145 +0000 UTC m=+893.114997524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-metrics-certs") pod "maas-controller-6886dd5f96-w5rhj" (UID: "f5b731af-c9bf-48c8-98dd-ad6becb06867") : secret "maas-controller-metrics-tls" not found Sep 09 21:16:16.761294 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.761268 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c9d897c74-gp2mx"] Sep 09 21:16:16.764987 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.764969 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c9d897c74-gp2mx" Sep 09 21:16:16.766752 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.766735 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf2m7\" (UniqueName: \"kubernetes.io/projected/f5b731af-c9bf-48c8-98dd-ad6becb06867-kube-api-access-qf2m7\") pod \"maas-controller-6886dd5f96-w5rhj\" (UID: \"f5b731af-c9bf-48c8-98dd-ad6becb06867\") " pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:16.782852 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.782824 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c9d897c74-gp2mx"] Sep 09 21:16:16.851334 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.851255 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:16.854333 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.854308 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88dfa476-fe4b-48b4-9bfa-f3def03d8057-metrics-certs\") pod \"maas-controller-5c9d897c74-gp2mx\" (UID: \"88dfa476-fe4b-48b4-9bfa-f3def03d8057\") " pod="opendatahub/maas-controller-5c9d897c74-gp2mx" Sep 09 21:16:16.854462 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.854356 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98ccj\" (UniqueName: \"kubernetes.io/projected/88dfa476-fe4b-48b4-9bfa-f3def03d8057-kube-api-access-98ccj\") pod \"maas-controller-5c9d897c74-gp2mx\" (UID: \"88dfa476-fe4b-48b4-9bfa-f3def03d8057\") " pod="opendatahub/maas-controller-5c9d897c74-gp2mx" Sep 09 21:16:16.854462 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.854439 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/88dfa476-fe4b-48b4-9bfa-f3def03d8057-webhook-cert\") pod \"maas-controller-5c9d897c74-gp2mx\" (UID: \"88dfa476-fe4b-48b4-9bfa-f3def03d8057\") " pod="opendatahub/maas-controller-5c9d897c74-gp2mx" Sep 09 21:16:16.955776 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.955738 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88dfa476-fe4b-48b4-9bfa-f3def03d8057-metrics-certs\") pod \"maas-controller-5c9d897c74-gp2mx\" (UID: \"88dfa476-fe4b-48b4-9bfa-f3def03d8057\") " pod="opendatahub/maas-controller-5c9d897c74-gp2mx" Sep 09 21:16:16.955968 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.955792 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98ccj\" (UniqueName: \"kubernetes.io/projected/88dfa476-fe4b-48b4-9bfa-f3def03d8057-kube-api-access-98ccj\") pod \"maas-controller-5c9d897c74-gp2mx\" (UID: \"88dfa476-fe4b-48b4-9bfa-f3def03d8057\") " pod="opendatahub/maas-controller-5c9d897c74-gp2mx" Sep 09 21:16:16.955968 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.955852 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/88dfa476-fe4b-48b4-9bfa-f3def03d8057-webhook-cert\") pod \"maas-controller-5c9d897c74-gp2mx\" (UID: \"88dfa476-fe4b-48b4-9bfa-f3def03d8057\") " pod="opendatahub/maas-controller-5c9d897c74-gp2mx" Sep 09 21:16:16.955968 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:16:16.955912 2560 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 09 21:16:16.956139 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:16:16.955977 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88dfa476-fe4b-48b4-9bfa-f3def03d8057-metrics-certs podName:88dfa476-fe4b-48b4-9bfa-f3def03d8057 nodeName:}" failed. No retries permitted until 2026-09-09 21:16:17.455962388 +0000 UTC m=+893.317484761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88dfa476-fe4b-48b4-9bfa-f3def03d8057-metrics-certs") pod "maas-controller-5c9d897c74-gp2mx" (UID: "88dfa476-fe4b-48b4-9bfa-f3def03d8057") : secret "maas-controller-metrics-tls" not found Sep 09 21:16:16.958495 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.958469 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/88dfa476-fe4b-48b4-9bfa-f3def03d8057-webhook-cert\") pod \"maas-controller-5c9d897c74-gp2mx\" (UID: \"88dfa476-fe4b-48b4-9bfa-f3def03d8057\") " pod="opendatahub/maas-controller-5c9d897c74-gp2mx" Sep 09 21:16:16.971570 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:16.971545 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98ccj\" (UniqueName: \"kubernetes.io/projected/88dfa476-fe4b-48b4-9bfa-f3def03d8057-kube-api-access-98ccj\") pod \"maas-controller-5c9d897c74-gp2mx\" (UID: \"88dfa476-fe4b-48b4-9bfa-f3def03d8057\") " pod="opendatahub/maas-controller-5c9d897c74-gp2mx" Sep 09 21:16:17.106442 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:17.106357 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:17.258772 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:17.258734 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-metrics-certs\") pod \"maas-controller-6886dd5f96-w5rhj\" (UID: \"f5b731af-c9bf-48c8-98dd-ad6becb06867\") " pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:17.258944 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:17.258793 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-webhook-cert\") pod \"maas-controller-6886dd5f96-w5rhj\" (UID: \"f5b731af-c9bf-48c8-98dd-ad6becb06867\") " pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:17.261303 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:17.261273 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-webhook-cert\") pod \"maas-controller-6886dd5f96-w5rhj\" (UID: \"f5b731af-c9bf-48c8-98dd-ad6becb06867\") " pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:17.261303 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:17.261299 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-metrics-certs\") pod \"maas-controller-6886dd5f96-w5rhj\" (UID: \"f5b731af-c9bf-48c8-98dd-ad6becb06867\") " pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:17.466319 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:17.461007 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88dfa476-fe4b-48b4-9bfa-f3def03d8057-metrics-certs\") pod \"maas-controller-5c9d897c74-gp2mx\" (UID: \"88dfa476-fe4b-48b4-9bfa-f3def03d8057\") " pod="opendatahub/maas-controller-5c9d897c74-gp2mx" Sep 09 21:16:17.466319 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:17.464153 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88dfa476-fe4b-48b4-9bfa-f3def03d8057-metrics-certs\") pod \"maas-controller-5c9d897c74-gp2mx\" (UID: \"88dfa476-fe4b-48b4-9bfa-f3def03d8057\") " pod="opendatahub/maas-controller-5c9d897c74-gp2mx" Sep 09 21:16:17.476762 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:17.476736 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:17.690179 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:17.690136 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c9d897c74-gp2mx" Sep 09 21:16:17.754931 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:17.754902 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6886dd5f96-w5rhj"] Sep 09 21:16:17.756560 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:16:17.756525 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b731af_c9bf_48c8_98dd_ad6becb06867.slice/crio-3657dce76f8d052d9c4fb94f44c5fda3c06f39f3407fce877b9204f5901b6e2d WatchSource:0}: Error finding container 3657dce76f8d052d9c4fb94f44c5fda3c06f39f3407fce877b9204f5901b6e2d: Status 404 returned error can't find the container with id 3657dce76f8d052d9c4fb94f44c5fda3c06f39f3407fce877b9204f5901b6e2d Sep 09 21:16:17.967797 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:17.967772 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c9d897c74-gp2mx"] Sep 09 21:16:17.969813 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:16:17.969784 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88dfa476_fe4b_48b4_9bfa_f3def03d8057.slice/crio-77529481c349763c86ec82e874d5b480f5952c601a8591bfd4fd8607d4173873 WatchSource:0}: Error finding container 77529481c349763c86ec82e874d5b480f5952c601a8591bfd4fd8607d4173873: Status 404 returned error can't find the container with id 77529481c349763c86ec82e874d5b480f5952c601a8591bfd4fd8607d4173873 Sep 09 21:16:18.431834 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:18.431793 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c9d897c74-gp2mx" event={"ID":"88dfa476-fe4b-48b4-9bfa-f3def03d8057","Type":"ContainerStarted","Data":"77529481c349763c86ec82e874d5b480f5952c601a8591bfd4fd8607d4173873"} Sep 09 21:16:18.433077 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:18.433031 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6886dd5f96-w5rhj" event={"ID":"f5b731af-c9bf-48c8-98dd-ad6becb06867","Type":"ContainerStarted","Data":"3657dce76f8d052d9c4fb94f44c5fda3c06f39f3407fce877b9204f5901b6e2d"} Sep 09 21:16:20.669270 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:20.669220 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:20.766464 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:20.766373 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:21.453978 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:21.453929 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6886dd5f96-w5rhj" event={"ID":"f5b731af-c9bf-48c8-98dd-ad6becb06867","Type":"ContainerStarted","Data":"e89f57bf6c571d20abf637080ed077302ef8555c77f171451bab95717a8563a1"} Sep 09 21:16:21.454216 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:21.454015 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:21.455794 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:21.455763 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c9d897c74-gp2mx" event={"ID":"88dfa476-fe4b-48b4-9bfa-f3def03d8057","Type":"ContainerStarted","Data":"e98471b6ee432c35d1dc7974de9599c2d83d70d4a42899413835404a8943270f"} Sep 09 21:16:21.455943 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:21.455926 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c9d897c74-gp2mx" Sep 09 21:16:21.481493 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:21.481443 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6886dd5f96-w5rhj" podStartSLOduration=2.7557356410000002 podStartE2EDuration="5.481428061s" totalImagesPullingTime="2.72569242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:16:21.479659035 +0000 UTC m=+897.341181429" watchObservedRunningTime="2026-09-09 21:16:21.481428061 +0000 UTC m=+897.342950456" Sep 09 21:16:21.504056 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:21.504001 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c9d897c74-gp2mx" podStartSLOduration=2.984351689 podStartE2EDuration="5.503986061s" totalImagesPullingTime="2.519634372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:16:21.503817778 +0000 UTC m=+897.365340174" watchObservedRunningTime="2026-09-09 21:16:21.503986061 +0000 UTC m=+897.365508456" Sep 09 21:16:32.463486 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:32.463447 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c9d897c74-gp2mx" Sep 09 21:16:32.463870 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:32.463617 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:32.544955 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:32.544915 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6886dd5f96-w5rhj"] Sep 09 21:16:32.545223 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:32.545197 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6886dd5f96-w5rhj" podUID="f5b731af-c9bf-48c8-98dd-ad6becb06867" containerName="manager" containerID="cri-o://e89f57bf6c571d20abf637080ed077302ef8555c77f171451bab95717a8563a1" gracePeriod=30 Sep 09 21:16:32.786246 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:32.786217 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:32.895411 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:32.895373 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qf2m7\" (UniqueName: \"kubernetes.io/projected/f5b731af-c9bf-48c8-98dd-ad6becb06867-kube-api-access-qf2m7\") pod \"f5b731af-c9bf-48c8-98dd-ad6becb06867\" (UID: \"f5b731af-c9bf-48c8-98dd-ad6becb06867\") " Sep 09 21:16:32.895411 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:32.895410 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-metrics-certs\") pod \"f5b731af-c9bf-48c8-98dd-ad6becb06867\" (UID: \"f5b731af-c9bf-48c8-98dd-ad6becb06867\") " Sep 09 21:16:32.895633 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:32.895492 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-webhook-cert\") pod \"f5b731af-c9bf-48c8-98dd-ad6becb06867\" (UID: \"f5b731af-c9bf-48c8-98dd-ad6becb06867\") " Sep 09 21:16:32.897789 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:32.897758 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f5b731af-c9bf-48c8-98dd-ad6becb06867" (UID: "f5b731af-c9bf-48c8-98dd-ad6becb06867"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:16:32.897789 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:32.897778 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f5b731af-c9bf-48c8-98dd-ad6becb06867" (UID: "f5b731af-c9bf-48c8-98dd-ad6becb06867"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:16:32.897947 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:32.897798 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5b731af-c9bf-48c8-98dd-ad6becb06867-kube-api-access-qf2m7" (OuterVolumeSpecName: "kube-api-access-qf2m7") pod "f5b731af-c9bf-48c8-98dd-ad6becb06867" (UID: "f5b731af-c9bf-48c8-98dd-ad6becb06867"). InnerVolumeSpecName "kube-api-access-qf2m7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:16:32.996182 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:32.996148 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qf2m7\" (UniqueName: \"kubernetes.io/projected/f5b731af-c9bf-48c8-98dd-ad6becb06867-kube-api-access-qf2m7\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:16:32.996182 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:32.996177 2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-metrics-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:16:32.996182 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:32.996186 2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5b731af-c9bf-48c8-98dd-ad6becb06867-webhook-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\"" Sep 09 21:16:33.523077 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:33.523027 2560 generic.go:356] "Generic (PLEG): container finished" podID="f5b731af-c9bf-48c8-98dd-ad6becb06867" containerID="e89f57bf6c571d20abf637080ed077302ef8555c77f171451bab95717a8563a1" exitCode=0 Sep 09 21:16:33.523516 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:33.523072 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6886dd5f96-w5rhj" event={"ID":"f5b731af-c9bf-48c8-98dd-ad6becb06867","Type":"ContainerDied","Data":"e89f57bf6c571d20abf637080ed077302ef8555c77f171451bab95717a8563a1"} Sep 09 21:16:33.523516 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:33.523125 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6886dd5f96-w5rhj" event={"ID":"f5b731af-c9bf-48c8-98dd-ad6becb06867","Type":"ContainerDied","Data":"3657dce76f8d052d9c4fb94f44c5fda3c06f39f3407fce877b9204f5901b6e2d"} Sep 09 21:16:33.523516 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:33.523147 2560 scope.go:117] "RemoveContainer" containerID="e89f57bf6c571d20abf637080ed077302ef8555c77f171451bab95717a8563a1" Sep 09 21:16:33.523516 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:33.523088 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6886dd5f96-w5rhj" Sep 09 21:16:33.532785 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:33.532768 2560 scope.go:117] "RemoveContainer" containerID="e89f57bf6c571d20abf637080ed077302ef8555c77f171451bab95717a8563a1" Sep 09 21:16:33.533085 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:16:33.533063 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e89f57bf6c571d20abf637080ed077302ef8555c77f171451bab95717a8563a1\": container with ID starting with e89f57bf6c571d20abf637080ed077302ef8555c77f171451bab95717a8563a1 not found: ID does not exist" containerID="e89f57bf6c571d20abf637080ed077302ef8555c77f171451bab95717a8563a1" Sep 09 21:16:33.533169 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:33.533097 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e89f57bf6c571d20abf637080ed077302ef8555c77f171451bab95717a8563a1"} err="failed to get container status \"e89f57bf6c571d20abf637080ed077302ef8555c77f171451bab95717a8563a1\": rpc error: code = NotFound desc = could not find container \"e89f57bf6c571d20abf637080ed077302ef8555c77f171451bab95717a8563a1\": container with ID starting with e89f57bf6c571d20abf637080ed077302ef8555c77f171451bab95717a8563a1 not found: ID does not exist" Sep 09 21:16:33.554364 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:33.552363 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6886dd5f96-w5rhj"] Sep 09 21:16:33.559281 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:33.559249 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6886dd5f96-w5rhj"] Sep 09 21:16:34.780213 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:34.780178 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5b731af-c9bf-48c8-98dd-ad6becb06867" path="/var/lib/kubelet/pods/f5b731af-c9bf-48c8-98dd-ad6becb06867/volumes" Sep 09 21:16:52.094546 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:52.094502 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:52.184463 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:52.184424 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:53.621230 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.621183 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:53.711824 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.711779 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:53.715866 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.715839 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6"] Sep 09 21:16:53.716209 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.716196 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5b731af-c9bf-48c8-98dd-ad6becb06867" containerName="manager" Sep 09 21:16:53.716262 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.716211 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5b731af-c9bf-48c8-98dd-ad6becb06867" containerName="manager" Sep 09 21:16:53.716299 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.716276 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5b731af-c9bf-48c8-98dd-ad6becb06867" containerName="manager" Sep 09 21:16:53.719132 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.719113 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:53.722321 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.722299 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 09 21:16:53.722449 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.722346 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 09 21:16:53.723194 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.723177 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hpvj8\"" Sep 09 21:16:53.735960 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.735926 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6"] Sep 09 21:16:53.759875 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.759837 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4e827c5-89bf-44d5-8f57-a9a1367035df-metrics-certs\") pod \"maas-api-7bf575b86-dxdv6\" (UID: \"b4e827c5-89bf-44d5-8f57-a9a1367035df\") " pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:53.760082 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.759900 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crqhv\" (UniqueName: \"kubernetes.io/projected/b4e827c5-89bf-44d5-8f57-a9a1367035df-kube-api-access-crqhv\") pod \"maas-api-7bf575b86-dxdv6\" (UID: \"b4e827c5-89bf-44d5-8f57-a9a1367035df\") " pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:53.760082 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.759939 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4e827c5-89bf-44d5-8f57-a9a1367035df-openshift-service-ca\") pod \"maas-api-7bf575b86-dxdv6\" (UID: \"b4e827c5-89bf-44d5-8f57-a9a1367035df\") " pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:53.760082 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.760009 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4e827c5-89bf-44d5-8f57-a9a1367035df-maas-api-tls\") pod \"maas-api-7bf575b86-dxdv6\" (UID: \"b4e827c5-89bf-44d5-8f57-a9a1367035df\") " pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:53.861182 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.861146 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crqhv\" (UniqueName: \"kubernetes.io/projected/b4e827c5-89bf-44d5-8f57-a9a1367035df-kube-api-access-crqhv\") pod \"maas-api-7bf575b86-dxdv6\" (UID: \"b4e827c5-89bf-44d5-8f57-a9a1367035df\") " pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:53.861373 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.861196 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4e827c5-89bf-44d5-8f57-a9a1367035df-openshift-service-ca\") pod \"maas-api-7bf575b86-dxdv6\" (UID: \"b4e827c5-89bf-44d5-8f57-a9a1367035df\") " pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:53.861373 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.861252 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4e827c5-89bf-44d5-8f57-a9a1367035df-maas-api-tls\") pod \"maas-api-7bf575b86-dxdv6\" (UID: \"b4e827c5-89bf-44d5-8f57-a9a1367035df\") " pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:53.861373 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.861305 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4e827c5-89bf-44d5-8f57-a9a1367035df-metrics-certs\") pod \"maas-api-7bf575b86-dxdv6\" (UID: \"b4e827c5-89bf-44d5-8f57-a9a1367035df\") " pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:53.861509 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:16:53.861410 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found Sep 09 21:16:53.861509 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:16:53.861502 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4e827c5-89bf-44d5-8f57-a9a1367035df-metrics-certs podName:b4e827c5-89bf-44d5-8f57-a9a1367035df nodeName:}" failed. No retries permitted until 2026-09-09 21:16:54.361468061 +0000 UTC m=+930.222990434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4e827c5-89bf-44d5-8f57-a9a1367035df-metrics-certs") pod "maas-api-7bf575b86-dxdv6" (UID: "b4e827c5-89bf-44d5-8f57-a9a1367035df") : secret "maas-api-metrics-tls" not found Sep 09 21:16:53.861962 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.861938 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4e827c5-89bf-44d5-8f57-a9a1367035df-openshift-service-ca\") pod \"maas-api-7bf575b86-dxdv6\" (UID: \"b4e827c5-89bf-44d5-8f57-a9a1367035df\") " pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:53.863922 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.863901 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4e827c5-89bf-44d5-8f57-a9a1367035df-maas-api-tls\") pod \"maas-api-7bf575b86-dxdv6\" (UID: \"b4e827c5-89bf-44d5-8f57-a9a1367035df\") " pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:53.874901 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.874835 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crqhv\" (UniqueName: \"kubernetes.io/projected/b4e827c5-89bf-44d5-8f57-a9a1367035df-kube-api-access-crqhv\") pod \"maas-api-7bf575b86-dxdv6\" (UID: \"b4e827c5-89bf-44d5-8f57-a9a1367035df\") " pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:53.904519 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:53.904485 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:54.008244 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.008203 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:54.350758 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.350719 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:54.365637 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.365602 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4e827c5-89bf-44d5-8f57-a9a1367035df-metrics-certs\") pod \"maas-api-7bf575b86-dxdv6\" (UID: \"b4e827c5-89bf-44d5-8f57-a9a1367035df\") " pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:54.368342 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.368228 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4e827c5-89bf-44d5-8f57-a9a1367035df-metrics-certs\") pod \"maas-api-7bf575b86-dxdv6\" (UID: \"b4e827c5-89bf-44d5-8f57-a9a1367035df\") " pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:54.447562 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.447523 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:54.624822 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.624741 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-j829s"] Sep 09 21:16:54.630091 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.630070 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-j829s" Sep 09 21:16:54.631120 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.631088 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:54.637595 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.637442 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 09 21:16:54.637768 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.637742 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zd5t2\"" Sep 09 21:16:54.659178 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.659141 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-j829s"] Sep 09 21:16:54.668686 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.668493 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55d8b13c-aba6-42b8-b800-7f38d31215c9-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-j829s\" (UID: \"55d8b13c-aba6-42b8-b800-7f38d31215c9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-j829s" Sep 09 21:16:54.668686 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.668578 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdjdn\" (UniqueName: \"kubernetes.io/projected/55d8b13c-aba6-42b8-b800-7f38d31215c9-kube-api-access-bdjdn\") pod \"payload-processing-5b54b7fbb7-j829s\" (UID: \"55d8b13c-aba6-42b8-b800-7f38d31215c9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-j829s" Sep 09 21:16:54.769359 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.769306 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55d8b13c-aba6-42b8-b800-7f38d31215c9-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-j829s\" (UID: \"55d8b13c-aba6-42b8-b800-7f38d31215c9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-j829s" Sep 09 21:16:54.769554 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.769380 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdjdn\" (UniqueName: \"kubernetes.io/projected/55d8b13c-aba6-42b8-b800-7f38d31215c9-kube-api-access-bdjdn\") pod \"payload-processing-5b54b7fbb7-j829s\" (UID: \"55d8b13c-aba6-42b8-b800-7f38d31215c9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-j829s" Sep 09 21:16:54.770361 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.770320 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55d8b13c-aba6-42b8-b800-7f38d31215c9-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-j829s\" (UID: \"55d8b13c-aba6-42b8-b800-7f38d31215c9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-j829s" Sep 09 21:16:54.791069 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.791022 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdjdn\" (UniqueName: \"kubernetes.io/projected/55d8b13c-aba6-42b8-b800-7f38d31215c9-kube-api-access-bdjdn\") pod \"payload-processing-5b54b7fbb7-j829s\" (UID: \"55d8b13c-aba6-42b8-b800-7f38d31215c9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-j829s" Sep 09 21:16:54.909671 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.909581 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:16:54.927978 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.927947 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6"] Sep 09 21:16:54.929036 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:16:54.929008 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e827c5_89bf_44d5_8f57_a9a1367035df.slice/crio-c6731a3a405e69fd77eea7df3e1e363f1556738a40f14b03d18dc502d2feffd7 WatchSource:0}: Error finding container c6731a3a405e69fd77eea7df3e1e363f1556738a40f14b03d18dc502d2feffd7: Status 404 returned error can't find the container with id c6731a3a405e69fd77eea7df3e1e363f1556738a40f14b03d18dc502d2feffd7 Sep 09 21:16:54.930324 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.930303 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:16:54.942458 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:54.942431 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-j829s" Sep 09 21:16:55.226268 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:55.226238 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-j829s"] Sep 09 21:16:55.228960 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:16:55.228925 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55d8b13c_aba6_42b8_b800_7f38d31215c9.slice/crio-1928cd9f9bb8d915222678e4c7f57ddd067729147be6a2d135c590a23b77bb9c WatchSource:0}: Error finding container 1928cd9f9bb8d915222678e4c7f57ddd067729147be6a2d135c590a23b77bb9c: Status 404 returned error can't find the container with id 1928cd9f9bb8d915222678e4c7f57ddd067729147be6a2d135c590a23b77bb9c Sep 09 21:16:55.644221 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:55.644118 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-j829s" event={"ID":"55d8b13c-aba6-42b8-b800-7f38d31215c9","Type":"ContainerStarted","Data":"1928cd9f9bb8d915222678e4c7f57ddd067729147be6a2d135c590a23b77bb9c"} Sep 09 21:16:55.645790 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:55.645759 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" event={"ID":"b4e827c5-89bf-44d5-8f57-a9a1367035df","Type":"ContainerStarted","Data":"c6731a3a405e69fd77eea7df3e1e363f1556738a40f14b03d18dc502d2feffd7"} Sep 09 21:16:57.665315 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:57.665270 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" event={"ID":"b4e827c5-89bf-44d5-8f57-a9a1367035df","Type":"ContainerStarted","Data":"f4f5235cc49072dee644e38a4f94cef40a3a03408b66e0df568599f500ce3754"} Sep 09 21:16:57.665797 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:57.665360 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:16:57.666857 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:57.666829 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-j829s" event={"ID":"55d8b13c-aba6-42b8-b800-7f38d31215c9","Type":"ContainerStarted","Data":"d618af9740515b67ed346e986693c55563faece62d6141facb98609d5e4c0a03"} Sep 09 21:16:57.666982 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:57.666889 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-j829s" Sep 09 21:16:57.695407 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:57.695353 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" podStartSLOduration=2.966102818 podStartE2EDuration="4.695336513s" totalImagesPullingTime="1.729233695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:16:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:16:57.692528392 +0000 UTC m=+933.554050786" watchObservedRunningTime="2026-09-09 21:16:57.695336513 +0000 UTC m=+933.556858907" Sep 09 21:16:57.719011 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:16:57.718951 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-j829s" podStartSLOduration=1.930096918 podStartE2EDuration="3.718932993s" totalImagesPullingTime="1.788836075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:16:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:16:57.717559639 +0000 UTC m=+933.579082034" watchObservedRunningTime="2026-09-09 21:16:57.718932993 +0000 UTC m=+933.580455387" Sep 09 21:17:03.678514 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:03.678484 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7bf575b86-dxdv6" Sep 09 21:17:04.802339 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:04.802303 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:17:04.900517 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:04.900479 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:17:08.675219 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:08.675188 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-j829s" Sep 09 21:17:13.949462 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:13.949428 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:17:15.224281 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:15.224244 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:17:15.332889 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:15.332831 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:17:27.305601 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.305560 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-2mllq"] Sep 09 21:17:27.310626 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.310602 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.313336 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.313289 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 09 21:17:27.313726 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.313698 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 09 21:17:27.313835 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.313741 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4pf52\"" Sep 09 21:17:27.313835 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.313750 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 09 21:17:27.323540 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.323513 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-2mllq"] Sep 09 21:17:27.396451 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.396417 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:17:27.442129 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.442093 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25dc1c7f-f482-4145-b242-44f114efe632-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.442298 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.442164 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25dc1c7f-f482-4145-b242-44f114efe632-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.442298 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.442208 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25dc1c7f-f482-4145-b242-44f114efe632-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.442298 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.442227 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq5r6\" (UniqueName: \"kubernetes.io/projected/25dc1c7f-f482-4145-b242-44f114efe632-kube-api-access-sq5r6\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.442425 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.442291 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25dc1c7f-f482-4145-b242-44f114efe632-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.442425 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.442367 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25dc1c7f-f482-4145-b242-44f114efe632-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.543639 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.543601 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25dc1c7f-f482-4145-b242-44f114efe632-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.543639 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.543644 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25dc1c7f-f482-4145-b242-44f114efe632-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.543878 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.543665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq5r6\" (UniqueName: \"kubernetes.io/projected/25dc1c7f-f482-4145-b242-44f114efe632-kube-api-access-sq5r6\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.543878 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.543854 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25dc1c7f-f482-4145-b242-44f114efe632-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.543988 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.543951 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25dc1c7f-f482-4145-b242-44f114efe632-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.544074 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.543993 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25dc1c7f-f482-4145-b242-44f114efe632-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.544185 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.544110 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25dc1c7f-f482-4145-b242-44f114efe632-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.544272 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.544250 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25dc1c7f-f482-4145-b242-44f114efe632-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.544325 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.544312 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25dc1c7f-f482-4145-b242-44f114efe632-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.546171 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.546144 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25dc1c7f-f482-4145-b242-44f114efe632-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.546286 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.546177 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25dc1c7f-f482-4145-b242-44f114efe632-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.553992 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.553970 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq5r6\" (UniqueName: \"kubernetes.io/projected/25dc1c7f-f482-4145-b242-44f114efe632-kube-api-access-sq5r6\") pod \"e2e-distinct-simulated-kserve-6685d86975-2mllq\" (UID: \"25dc1c7f-f482-4145-b242-44f114efe632\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.625007 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.624917 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:27.906111 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:27.906079 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-2mllq"] Sep 09 21:17:27.906930 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:17:27.906907 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25dc1c7f_f482_4145_b242_44f114efe632.slice/crio-37cf78bd279d6b5a515eb885978cae3bc48d0078a9c10cc3ffc538874e1aae26 WatchSource:0}: Error finding container 37cf78bd279d6b5a515eb885978cae3bc48d0078a9c10cc3ffc538874e1aae26: Status 404 returned error can't find the container with id 37cf78bd279d6b5a515eb885978cae3bc48d0078a9c10cc3ffc538874e1aae26 Sep 09 21:17:28.841894 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:28.841850 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" event={"ID":"25dc1c7f-f482-4145-b242-44f114efe632","Type":"ContainerStarted","Data":"37cf78bd279d6b5a515eb885978cae3bc48d0078a9c10cc3ffc538874e1aae26"} Sep 09 21:17:29.849486 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:29.849446 2560 generic.go:356] "Generic (PLEG): container finished" podID="25dc1c7f-f482-4145-b242-44f114efe632" containerID="9ee836953553557b21c18bbfbaf10bbd2880b0dab3c602816848c49279e596c2" exitCode=0 Sep 09 21:17:29.849906 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:29.849532 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" event={"ID":"25dc1c7f-f482-4145-b242-44f114efe632","Type":"ContainerDied","Data":"9ee836953553557b21c18bbfbaf10bbd2880b0dab3c602816848c49279e596c2"} Sep 09 21:17:29.849906 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:29.849757 2560 scope.go:117] "RemoveContainer" containerID="9ee836953553557b21c18bbfbaf10bbd2880b0dab3c602816848c49279e596c2" Sep 09 21:17:30.857171 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:30.857134 2560 generic.go:356] "Generic (PLEG): container finished" podID="25dc1c7f-f482-4145-b242-44f114efe632" containerID="04d80acacb909ac7e0d1723b7ef890344e1c57aa9637a133a23c15573ec9802e" exitCode=0 Sep 09 21:17:30.857611 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:30.857217 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" event={"ID":"25dc1c7f-f482-4145-b242-44f114efe632","Type":"ContainerDied","Data":"04d80acacb909ac7e0d1723b7ef890344e1c57aa9637a133a23c15573ec9802e"} Sep 09 21:17:30.857611 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:30.857277 2560 scope.go:117] "RemoveContainer" containerID="9ee836953553557b21c18bbfbaf10bbd2880b0dab3c602816848c49279e596c2" Sep 09 21:17:30.857611 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:30.857514 2560 scope.go:117] "RemoveContainer" containerID="04d80acacb909ac7e0d1723b7ef890344e1c57aa9637a133a23c15573ec9802e" Sep 09 21:17:30.857731 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:17:30.857706 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:17:31.864653 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:31.864618 2560 scope.go:117] "RemoveContainer" containerID="04d80acacb909ac7e0d1723b7ef890344e1c57aa9637a133a23c15573ec9802e" Sep 09 21:17:31.865035 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:17:31.864800 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:17:36.500781 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:36.500732 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:17:36.606322 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:36.606291 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:17:37.625120 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:37.625084 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:37.625512 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:37.625149 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:37.625512 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:37.625165 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:37.625512 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:37.625182 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:37.625626 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:37.625611 2560 scope.go:117] "RemoveContainer" containerID="04d80acacb909ac7e0d1723b7ef890344e1c57aa9637a133a23c15573ec9802e" Sep 09 21:17:37.625798 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:17:37.625778 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:17:44.505657 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:44.505616 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:17:49.774997 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:49.774910 2560 scope.go:117] "RemoveContainer" containerID="04d80acacb909ac7e0d1723b7ef890344e1c57aa9637a133a23c15573ec9802e" Sep 09 21:17:50.971230 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:50.971193 2560 generic.go:356] "Generic (PLEG): container finished" podID="25dc1c7f-f482-4145-b242-44f114efe632" containerID="869404f4c0c636ddb977d12de38323c2b68f7b9494b619131dfd15dc4a475a39" exitCode=0 Sep 09 21:17:50.971631 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:50.971270 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" event={"ID":"25dc1c7f-f482-4145-b242-44f114efe632","Type":"ContainerDied","Data":"869404f4c0c636ddb977d12de38323c2b68f7b9494b619131dfd15dc4a475a39"} Sep 09 21:17:50.971631 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:50.971337 2560 scope.go:117] "RemoveContainer" containerID="04d80acacb909ac7e0d1723b7ef890344e1c57aa9637a133a23c15573ec9802e" Sep 09 21:17:50.971726 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:50.971687 2560 scope.go:117] "RemoveContainer" containerID="869404f4c0c636ddb977d12de38323c2b68f7b9494b619131dfd15dc4a475a39" Sep 09 21:17:50.971888 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:17:50.971864 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:17:54.435397 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:54.435354 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:17:57.626108 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:57.626066 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:57.626535 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:57.626164 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:57.626535 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:57.626184 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:57.626535 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:57.626198 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:17:57.626646 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:17:57.626582 2560 scope.go:117] "RemoveContainer" containerID="869404f4c0c636ddb977d12de38323c2b68f7b9494b619131dfd15dc4a475a39" Sep 09 21:17:57.626784 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:17:57.626763 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:18:10.774644 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:10.774615 2560 scope.go:117] "RemoveContainer" containerID="869404f4c0c636ddb977d12de38323c2b68f7b9494b619131dfd15dc4a475a39" Sep 09 21:18:11.083375 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:11.083342 2560 generic.go:356] "Generic (PLEG): container finished" podID="25dc1c7f-f482-4145-b242-44f114efe632" containerID="4824ce22ec848a6d5a46b3114936a3b4922b1db5f51e0f5ae9b952e987ff0dc8" exitCode=0 Sep 09 21:18:11.083512 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:11.083414 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" event={"ID":"25dc1c7f-f482-4145-b242-44f114efe632","Type":"ContainerDied","Data":"4824ce22ec848a6d5a46b3114936a3b4922b1db5f51e0f5ae9b952e987ff0dc8"} Sep 09 21:18:11.083560 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:11.083509 2560 scope.go:117] "RemoveContainer" containerID="869404f4c0c636ddb977d12de38323c2b68f7b9494b619131dfd15dc4a475a39" Sep 09 21:18:11.083872 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:11.083851 2560 scope.go:117] "RemoveContainer" containerID="4824ce22ec848a6d5a46b3114936a3b4922b1db5f51e0f5ae9b952e987ff0dc8" Sep 09 21:18:11.084048 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:18:11.084022 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:18:13.266512 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:13.266477 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:18:17.625309 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:17.625272 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:18:17.625773 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:17.625333 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:18:17.625773 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:17.625351 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:18:17.625773 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:17.625365 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:18:17.625773 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:17.625672 2560 scope.go:117] "RemoveContainer" containerID="4824ce22ec848a6d5a46b3114936a3b4922b1db5f51e0f5ae9b952e987ff0dc8" Sep 09 21:18:17.625904 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:18:17.625824 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:18:20.931173 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:20.931140 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:18:28.780445 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:28.780416 2560 scope.go:117] "RemoveContainer" containerID="4824ce22ec848a6d5a46b3114936a3b4922b1db5f51e0f5ae9b952e987ff0dc8" Sep 09 21:18:28.780960 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:18:28.780599 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:18:39.774815 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:39.774779 2560 scope.go:117] "RemoveContainer" containerID="4824ce22ec848a6d5a46b3114936a3b4922b1db5f51e0f5ae9b952e987ff0dc8" Sep 09 21:18:39.775315 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:18:39.774942 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:18:52.774145 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:52.774109 2560 scope.go:117] "RemoveContainer" containerID="4824ce22ec848a6d5a46b3114936a3b4922b1db5f51e0f5ae9b952e987ff0dc8" Sep 09 21:18:53.310816 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:53.310722 2560 generic.go:356] "Generic (PLEG): container finished" podID="25dc1c7f-f482-4145-b242-44f114efe632" containerID="a9005fdd0b71a4c1f66970e5fd854becb13b374d93b2ea8298cb76ce04ef05b3" exitCode=0 Sep 09 21:18:53.310816 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:53.310761 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" event={"ID":"25dc1c7f-f482-4145-b242-44f114efe632","Type":"ContainerDied","Data":"a9005fdd0b71a4c1f66970e5fd854becb13b374d93b2ea8298cb76ce04ef05b3"} Sep 09 21:18:53.310816 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:53.310812 2560 scope.go:117] "RemoveContainer" containerID="4824ce22ec848a6d5a46b3114936a3b4922b1db5f51e0f5ae9b952e987ff0dc8" Sep 09 21:18:53.311203 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:53.311182 2560 scope.go:117] "RemoveContainer" containerID="a9005fdd0b71a4c1f66970e5fd854becb13b374d93b2ea8298cb76ce04ef05b3" Sep 09 21:18:53.311428 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:18:53.311405 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:18:57.625464 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:57.625425 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:18:57.625863 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:57.625493 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:18:57.625863 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:57.625509 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:18:57.625863 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:57.625527 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:18:57.625863 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:18:57.625849 2560 scope.go:117] "RemoveContainer" containerID="a9005fdd0b71a4c1f66970e5fd854becb13b374d93b2ea8298cb76ce04ef05b3" Sep 09 21:18:57.626027 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:18:57.626009 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:19:04.820541 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:19:04.820506 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:19:06.718843 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:19:06.718804 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:19:08.774225 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:19:08.774191 2560 scope.go:117] "RemoveContainer" containerID="a9005fdd0b71a4c1f66970e5fd854becb13b374d93b2ea8298cb76ce04ef05b3" Sep 09 21:19:08.774618 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:19:08.774380 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:19:21.774839 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:19:21.774762 2560 scope.go:117] "RemoveContainer" containerID="a9005fdd0b71a4c1f66970e5fd854becb13b374d93b2ea8298cb76ce04ef05b3" Sep 09 21:19:21.775297 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:19:21.774983 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:19:33.774759 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:19:33.774727 2560 scope.go:117] "RemoveContainer" containerID="a9005fdd0b71a4c1f66970e5fd854becb13b374d93b2ea8298cb76ce04ef05b3" Sep 09 21:19:33.775258 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:19:33.774914 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:19:42.701012 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:19:42.700971 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:19:46.775007 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:19:46.774972 2560 scope.go:117] "RemoveContainer" containerID="a9005fdd0b71a4c1f66970e5fd854becb13b374d93b2ea8298cb76ce04ef05b3" Sep 09 21:19:46.775409 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:19:46.775213 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:19:59.774360 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:19:59.774321 2560 scope.go:117] "RemoveContainer" containerID="a9005fdd0b71a4c1f66970e5fd854becb13b374d93b2ea8298cb76ce04ef05b3" Sep 09 21:19:59.774762 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:19:59.774487 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:20:13.774249 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:20:13.774216 2560 scope.go:117] "RemoveContainer" containerID="a9005fdd0b71a4c1f66970e5fd854becb13b374d93b2ea8298cb76ce04ef05b3" Sep 09 21:20:14.745355 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:20:14.745321 2560 generic.go:356] "Generic (PLEG): container finished" podID="25dc1c7f-f482-4145-b242-44f114efe632" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" exitCode=0 Sep 09 21:20:14.745556 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:20:14.745399 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" event={"ID":"25dc1c7f-f482-4145-b242-44f114efe632","Type":"ContainerDied","Data":"ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17"} Sep 09 21:20:14.745556 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:20:14.745473 2560 scope.go:117] "RemoveContainer" containerID="a9005fdd0b71a4c1f66970e5fd854becb13b374d93b2ea8298cb76ce04ef05b3" Sep 09 21:20:14.745863 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:20:14.745847 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:20:14.746114 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:20:14.746096 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:20:17.625424 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:20:17.625378 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:20:17.625424 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:20:17.625440 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:20:17.625872 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:20:17.625457 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:20:17.625872 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:20:17.625471 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" Sep 09 21:20:17.625872 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:20:17.625758 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:20:17.625969 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:20:17.625915 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:20:17.762346 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:20:17.762308 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:20:17.762568 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:20:17.762541 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:20:32.780867 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:20:32.780836 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:20:32.781346 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:20:32.781025 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:20:46.774832 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:20:46.774798 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:20:46.775291 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:20:46.774971 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:21:01.774170 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:21:01.774139 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:21:01.774649 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:21:01.774309 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:21:14.776657 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:21:14.776619 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:21:14.777188 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:21:14.776806 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:21:25.774803 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:21:25.774769 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:21:25.775388 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:21:25.774941 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:21:35.550831 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:21:35.550790 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:21:36.774498 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:21:36.774463 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:21:36.775000 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:21:36.774650 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:21:48.774726 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:21:48.774693 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:21:48.775170 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:21:48.774860 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:22:03.774469 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:03.774434 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:22:03.774891 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:22:03.774630 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:22:10.071482 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:10.071448 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84cbdc795c-dmh52_58226d15-493f-4037-8628-e82e51578987/authorino/0.log" Sep 09 21:22:15.774966 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:15.774928 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:22:15.775382 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:22:15.775208 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:22:15.932866 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:15.932835 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8bh26_d915fcaa-3a64-40d0-917f-b07ee74e8a53/manager/0.log" Sep 09 21:22:16.054651 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:16.054561 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mb75z_e856b08f-2e82-4dd4-a929-ba32adcc282a/manager/0.log" Sep 09 21:22:16.182874 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:16.182848 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c9d897c74-gp2mx_88dfa476-fe4b-48b4-9bfa-f3def03d8057/manager/0.log" Sep 09 21:22:16.506232 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:16.506189 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:22:16.843804 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:16.843686 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59685ff6b-jctfr_d9c661f8-fbaf-40a3-82c1-401cfa47885b/copy-manifests/0.log" Sep 09 21:22:16.858137 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:16.858103 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59685ff6b-jctfr_d9c661f8-fbaf-40a3-82c1-401cfa47885b/manager/0.log" Sep 09 21:22:17.226450 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:17.226418 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7bf575b86-dxdv6_b4e827c5-89bf-44d5-8f57-a9a1367035df/maas-api/0.log" Sep 09 21:22:17.362680 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:17.362640 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-zmhs4_c8d843b5-e932-44fa-825a-46f1c81fd4c6/postgres/0.log" Sep 09 21:22:18.220375 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:18.220341 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84cbdc795c-dmh52_58226d15-493f-4037-8628-e82e51578987/authorino/0.log" Sep 09 21:22:18.338107 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:18.338083 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-8kdnh_e1c77a06-4045-4d55-a5d3-0a50b1f3d8ec/manager/0.log" Sep 09 21:22:18.454414 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:18.454376 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-pkcxf_99925480-9343-4aaa-990e-75c2b56399a0/manager/0.log" Sep 09 21:22:18.577430 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:18.577353 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-zgg7l_9967ce24-503f-4981-b9ee-c71a80beaadd/kuadrant-console-plugin/0.log" Sep 09 21:22:18.711363 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:18.711314 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-gbmgd_89252b48-4768-494e-826d-7f9ddd24bb0b/manager/0.log" Sep 09 21:22:18.958659 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:18.958632 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-76knv_59118908-dd81-40c1-ba8c-6beff25aaa9f/manager/0.log" Sep 09 21:22:19.565736 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:19.565704 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f7b6bff6-6mchv_84a4f911-9970-4e19-be48-6c6beff033a8/kube-auth-proxy/0.log" Sep 09 21:22:20.064055 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:20.064020 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-j829s_55d8b13c-aba6-42b8-b800-7f38d31215c9/payload-processing/0.log" Sep 09 21:22:20.663863 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:20.663833 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-6685d86975-2mllq_25dc1c7f-f482-4145-b242-44f114efe632/main/5.log" Sep 09 21:22:26.351769 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:26.351729 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:22:26.447732 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:26.447695 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:22:26.774284 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:26.774254 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:22:26.774450 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:22:26.774428 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:22:27.104774 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:27.104676 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:22:27.180551 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:27.180509 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" limit=1024 Sep 09 21:22:31.107019 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:31.106968 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gj6c4_253428b3-7b9b-474c-b3eb-f8b8f63b9671/global-pull-secret-syncer/0.log" Sep 09 21:22:31.400534 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:31.400441 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-472cd_e1b87a69-3799-43a2-8e4e-deffdbe8ae09/konnectivity-agent/0.log" Sep 09 21:22:31.587636 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:31.587608 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-73.ec2.internal_1ea201ea53f48274b482ba7553442dfe/haproxy/0.log" Sep 09 21:22:35.158609 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:35.158576 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84cbdc795c-dmh52_58226d15-493f-4037-8628-e82e51578987/authorino/0.log" Sep 09 21:22:35.214716 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:35.214686 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-8kdnh_e1c77a06-4045-4d55-a5d3-0a50b1f3d8ec/manager/0.log" Sep 09 21:22:35.271669 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:35.271643 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-pkcxf_99925480-9343-4aaa-990e-75c2b56399a0/manager/0.log" Sep 09 21:22:35.368854 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:35.368825 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-zgg7l_9967ce24-503f-4981-b9ee-c71a80beaadd/kuadrant-console-plugin/0.log" Sep 09 21:22:35.414646 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:35.414545 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-gbmgd_89252b48-4768-494e-826d-7f9ddd24bb0b/manager/0.log" Sep 09 21:22:35.510937 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:35.510899 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-76knv_59118908-dd81-40c1-ba8c-6beff25aaa9f/manager/0.log" Sep 09 21:22:37.634666 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:37.634634 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g4nlp_87c8eb5c-6363-4821-adc5-cc6eb11a91c4/node-exporter/0.log" Sep 09 21:22:37.661133 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:37.661093 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g4nlp_87c8eb5c-6363-4821-adc5-cc6eb11a91c4/kube-rbac-proxy/0.log" Sep 09 21:22:37.688880 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:37.688851 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g4nlp_87c8eb5c-6363-4821-adc5-cc6eb11a91c4/init-textfile/0.log" Sep 09 21:22:40.461786 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:40.461753 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-6jqtn_9a130f42-a944-429a-b888-56792ed2d998/networking-console-plugin/0.log" Sep 09 21:22:40.775091 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:40.774986 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:22:40.775237 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:22:40.775207 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:22:41.385055 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.385019 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr"] Sep 09 21:22:41.387457 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.387437 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.390416 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.390393 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dvsfq\"/\"default-dockercfg-rmncw\"" Sep 09 21:22:41.390514 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.390482 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dvsfq\"/\"openshift-service-ca.crt\"" Sep 09 21:22:41.391389 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.391374 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dvsfq\"/\"kube-root-ca.crt\"" Sep 09 21:22:41.401700 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.401678 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr"] Sep 09 21:22:41.505677 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.505641 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-podres\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.505677 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.505679 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-proc\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.506133 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.505698 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-sys\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.506133 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.505810 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz7vl\" (UniqueName: \"kubernetes.io/projected/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-kube-api-access-jz7vl\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.506133 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.505844 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-lib-modules\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.606500 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.606459 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz7vl\" (UniqueName: \"kubernetes.io/projected/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-kube-api-access-jz7vl\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.606500 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.606505 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-lib-modules\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.606730 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.606540 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-podres\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.606730 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.606556 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-proc\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.606730 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.606577 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-sys\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.606730 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.606670 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-sys\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.606730 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.606671 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-proc\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.606730 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.606715 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-podres\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.606920 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.606726 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-lib-modules\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.616543 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.616517 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz7vl\" (UniqueName: \"kubernetes.io/projected/7c88dcf0-6cbb-483d-b1ee-631d491cd0e4-kube-api-access-jz7vl\") pod \"perf-node-gather-daemonset-hsmlr\" (UID: \"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.697692 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.697604 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:41.975947 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:41.975923 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr"] Sep 09 21:22:41.977654 ip-10-0-140-73 kubenswrapper[2560]: W0909 21:22:41.977627 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7c88dcf0_6cbb_483d_b1ee_631d491cd0e4.slice/crio-8737f71380c90947d7db3ef76181c0c1cc8fff45a2881f548a81a9878fe106f8 WatchSource:0}: Error finding container 8737f71380c90947d7db3ef76181c0c1cc8fff45a2881f548a81a9878fe106f8: Status 404 returned error can't find the container with id 8737f71380c90947d7db3ef76181c0c1cc8fff45a2881f548a81a9878fe106f8 Sep 09 21:22:42.525258 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:42.525220 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" event={"ID":"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4","Type":"ContainerStarted","Data":"8cb003f38390f8f1b9375ec9ec58a686982b0bf4654b4d0b8bdddc89d609ca47"} Sep 09 21:22:42.525761 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:42.525270 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" event={"ID":"7c88dcf0-6cbb-483d-b1ee-631d491cd0e4","Type":"ContainerStarted","Data":"8737f71380c90947d7db3ef76181c0c1cc8fff45a2881f548a81a9878fe106f8"} Sep 09 21:22:42.525761 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:42.525356 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:42.545685 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:42.545637 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" podStartSLOduration=1.545621788 podStartE2EDuration="1.545621788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:22:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:22:42.543607565 +0000 UTC m=+1278.405129959" watchObservedRunningTime="2026-09-09 21:22:42.545621788 +0000 UTC m=+1278.407144183" Sep 09 21:22:43.682567 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:43.682492 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5j4zn_f73f9bb9-8de7-4211-a99d-758f7c2721b5/dns/0.log" Sep 09 21:22:43.715676 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:43.715648 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5j4zn_f73f9bb9-8de7-4211-a99d-758f7c2721b5/kube-rbac-proxy/0.log" Sep 09 21:22:43.871761 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:43.871729 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jq8n4_71b2bab2-31a6-4b4b-9f09-44ffbfd210c6/dns-node-resolver/0.log" Sep 09 21:22:44.705372 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:44.705341 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sdc89_76b95cc0-0a15-4c20-bd29-463861438b51/node-ca/0.log" Sep 09 21:22:45.881076 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:45.881027 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f7b6bff6-6mchv_84a4f911-9970-4e19-be48-6c6beff033a8/kube-auth-proxy/0.log" Sep 09 21:22:46.006724 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:46.006696 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-j829s_55d8b13c-aba6-42b8-b800-7f38d31215c9/payload-processing/0.log" Sep 09 21:22:46.759623 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:46.759596 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nv9wb_588a22bb-35b2-43cc-86fa-50afff61eb82/serve-healthcheck-canary/0.log" Sep 09 21:22:47.401669 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:47.401634 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-rrrx7_2da07b2d-4244-45d8-87e5-ada3a82297fc/insights-operator/1.log" Sep 09 21:22:47.402093 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:47.401731 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-rrrx7_2da07b2d-4244-45d8-87e5-ada3a82297fc/insights-operator/0.log" Sep 09 21:22:47.671593 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:47.671517 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d99k5_af7ee458-4682-48c8-90d3-c2c2673ed791/kube-rbac-proxy/0.log" Sep 09 21:22:47.701622 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:47.701592 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d99k5_af7ee458-4682-48c8-90d3-c2c2673ed791/exporter/0.log" Sep 09 21:22:47.728979 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:47.728949 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d99k5_af7ee458-4682-48c8-90d3-c2c2673ed791/extractor/0.log" Sep 09 21:22:48.540433 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:48.540407 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-hsmlr" Sep 09 21:22:50.092316 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:50.092286 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8bh26_d915fcaa-3a64-40d0-917f-b07ee74e8a53/manager/0.log" Sep 09 21:22:50.137129 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:50.137094 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mb75z_e856b08f-2e82-4dd4-a929-ba32adcc282a/manager/0.log" Sep 09 21:22:50.186551 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:50.186522 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c9d897c74-gp2mx_88dfa476-fe4b-48b4-9bfa-f3def03d8057/manager/0.log" Sep 09 21:22:50.505638 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:50.505609 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59685ff6b-jctfr_d9c661f8-fbaf-40a3-82c1-401cfa47885b/manager/0.log" Sep 09 21:22:50.529602 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:50.529576 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59685ff6b-jctfr_d9c661f8-fbaf-40a3-82c1-401cfa47885b/copy-manifests/0.log" Sep 09 21:22:51.581444 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:51.581413 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f7b6bff6-6mchv_84a4f911-9970-4e19-be48-6c6beff033a8/kube-auth-proxy/0.log" Sep 09 21:22:51.678469 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:51.678389 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-j829s_55d8b13c-aba6-42b8-b800-7f38d31215c9/payload-processing/0.log" Sep 09 21:22:52.774569 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:52.774538 2560 scope.go:117] "RemoveContainer" containerID="ed19733483bc1639f35e8325307211a4cdbc58b89e0dd59cc274a423273f7b17" Sep 09 21:22:52.774976 ip-10-0-140-73 kubenswrapper[2560]: E0909 21:22:52.774758 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-2mllq_llm(25dc1c7f-f482-4145-b242-44f114efe632)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-2mllq" podUID="25dc1c7f-f482-4145-b242-44f114efe632" Sep 09 21:22:53.128020 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:53.127941 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f7b6bff6-6mchv_84a4f911-9970-4e19-be48-6c6beff033a8/kube-auth-proxy/0.log" Sep 09 21:22:53.233502 ip-10-0-140-73 kubenswrapper[2560]: I0909 21:22:53.233473 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-j829s_55d8b13c-aba6-42b8-b800-7f38d31215c9/payload-processing/0.log"