Jul 09 03:40:31.505201 ip-10-0-137-230 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 03:40:31.505217 ip-10-0-137-230 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 03:40:31.505227 ip-10-0-137-230 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 03:40:31.505629 ip-10-0-137-230 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 03:40:41.048289 ip-10-0-137-230 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 09 03:40:41.048300 ip-10-0-137-230 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 81288d3d282d469c8b37b4d942ea2027 -- Jul 09 03:42:09.230951 ip-10-0-137-230 systemd[1]: Starting Kubernetes Kubelet... Jul 09 03:42:09.789119 ip-10-0-137-230 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 03:42:09.789119 ip-10-0-137-230 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 03:42:09.789119 ip-10-0-137-230 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 03:42:09.789119 ip-10-0-137-230 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 03:42:09.789119 ip-10-0-137-230 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 03:42:09.789119 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.688922 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 03:42:09.789119 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694616 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 03:42:09.789119 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694634 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 03:42:09.789119 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694638 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 03:42:09.789119 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694641 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 03:42:09.789119 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694656 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 03:42:09.789119 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694659 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 03:42:09.789119 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694663 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 03:42:09.789119 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694666 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694669 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694672 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694675 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694677 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694681 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694683 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694686 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694697 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694700 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694703 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694706 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694709 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694712 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694715 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694717 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694720 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694723 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694726 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694728 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 03:42:09.793419 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694731 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 03:42:09.776700 ip-10-0-137-230 systemd[1]: Started Kubernetes Kubelet. Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694734 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694736 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694739 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694742 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694745 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694747 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694750 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694752 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694755 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694757 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694760 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694762 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694764 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694767 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694769 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694772 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694774 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694777 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 03:42:09.796906 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694779 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694782 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694785 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694788 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694790 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694795 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694799 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694803 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694807 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694810 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694814 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694817 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694820 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694822 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694826 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694829 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694832 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694835 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694837 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 03:42:09.799858 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694840 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694843 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694847 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694850 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694853 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694856 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694858 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694861 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694863 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694866 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694870 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694872 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694875 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694878 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694882 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694885 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694888 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694891 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694894 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694896 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 03:42:09.802200 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694900 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694902 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694905 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694907 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694910 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694912 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694915 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694917 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694920 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694922 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694925 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694927 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694930 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.694932 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695685 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695692 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695696 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695699 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695701 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695703 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 03:42:09.804676 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695706 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695709 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695711 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695714 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695718 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695721 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695724 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695727 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695730 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695733 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695737 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695740 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695742 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695745 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695747 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695750 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695753 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695755 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695758 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695761 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 03:42:09.806762 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695763 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695766 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695768 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695772 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695774 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695777 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695780 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695783 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695785 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695788 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695791 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695793 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695796 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695799 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695801 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695804 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695806 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695809 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695813 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695817 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 03:42:09.821521 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695820 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695823 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695826 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695830 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695834 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695837 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695840 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695843 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695845 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695848 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695851 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695855 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695858 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695861 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695863 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695866 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695869 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695872 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695876 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 03:42:09.837140 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695879 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695881 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695884 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695886 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695892 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695894 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695898 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695900 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695903 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695905 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695908 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695911 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695913 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695916 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695919 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695921 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695925 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695927 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695930 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695932 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 03:42:09.840212 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695935 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695937 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695940 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695942 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695945 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695948 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695950 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695953 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695956 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695959 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695961 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695964 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695967 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.695969 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696378 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696383 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696388 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696391 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696394 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696396 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 03:42:09.990017 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696399 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696402 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696405 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696408 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696410 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696413 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696415 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696418 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696421 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696424 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696427 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696430 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696432 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696435 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696437 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696440 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696442 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696445 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696448 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696451 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 03:42:10.009387 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696453 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696456 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696459 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696461 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696463 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696466 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696468 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696471 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696474 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696477 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696479 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696482 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696485 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696488 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696490 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696493 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696496 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696499 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696501 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696504 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 03:42:10.010138 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696506 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696510 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696512 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696518 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696520 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696523 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696525 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696528 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696530 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696533 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696535 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696538 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696540 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696543 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696545 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696548 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696550 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696553 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696555 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696558 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 03:42:10.011035 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696562 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696565 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696567 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696570 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696572 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696576 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696578 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696581 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696583 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696586 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696588 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696590 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696593 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696595 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696598 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696601 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696604 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696606 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696609 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 03:42:10.012608 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696613 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696616 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696619 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696622 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696626 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696630 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696634 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696638 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696641 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696658 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696661 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696664 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696666 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.696682 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696760 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696768 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696775 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696780 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696785 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696788 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 03:42:10.014354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696793 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696798 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696801 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696804 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696808 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696811 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696814 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696817 2566 flags.go:64] FLAG: --cgroup-root="" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696823 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696826 2566 flags.go:64] FLAG: --client-ca-file="" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696829 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696832 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696836 2566 flags.go:64] FLAG: --cluster-domain="" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696839 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696842 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696845 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696848 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696852 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696855 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696858 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696861 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696864 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696867 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696870 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 03:42:10.015022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696873 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696878 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696882 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696885 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696887 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696890 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696893 2566 flags.go:64] FLAG: --enable-server="true" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696896 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696901 2566 flags.go:64] FLAG: --event-burst="100" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696904 2566 flags.go:64] FLAG: --event-qps="50" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696908 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696910 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696913 2566 flags.go:64] FLAG: --eviction-hard="" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696917 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696920 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696923 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696927 2566 flags.go:64] FLAG: --eviction-soft="" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696931 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696934 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696937 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696940 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696943 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696946 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696949 2566 flags.go:64] FLAG: --feature-gates="" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696953 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 03:42:10.017147 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696957 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696960 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696963 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696966 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696969 2566 flags.go:64] FLAG: --help="false" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696972 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696975 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696978 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696982 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696985 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696989 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696992 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696995 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.696998 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697001 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697003 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697007 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697009 2566 flags.go:64] FLAG: --kube-reserved="" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697012 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697015 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697018 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697021 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697024 2566 flags.go:64] FLAG: --lock-file="" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697027 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 03:42:10.019483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697031 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697035 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697040 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697043 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697046 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697049 2566 flags.go:64] FLAG: --logging-format="text" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697052 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697055 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697058 2566 flags.go:64] FLAG: --manifest-url="" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697060 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697065 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697068 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697072 2566 flags.go:64] FLAG: --max-pods="110" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697075 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697078 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697080 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697085 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697088 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697091 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697094 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697101 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697104 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697107 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 03:42:10.021584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697110 2566 flags.go:64] FLAG: --pod-cidr="" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697113 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697119 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697121 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697124 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697127 2566 flags.go:64] FLAG: --port="10250" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697130 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697133 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f206702b1497ca03" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697137 2566 flags.go:64] FLAG: --qos-reserved="" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697141 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697144 2566 flags.go:64] FLAG: --register-node="true" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697149 2566 flags.go:64] FLAG: --register-with-taints="" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697153 2566 flags.go:64] FLAG: --registry-burst="10" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697156 2566 flags.go:64] FLAG: --registry-qps="5" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697159 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697161 2566 flags.go:64] FLAG: --reserved-memory="" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697165 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697168 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697171 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697174 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697177 2566 flags.go:64] FLAG: --runonce="false" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697180 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697183 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697186 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697189 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 03:42:10.027422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697194 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697197 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697200 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697203 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697206 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697209 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697212 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697214 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697218 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697221 2566 flags.go:64] FLAG: --system-cgroups="" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697224 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697229 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697231 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697234 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697238 2566 flags.go:64] FLAG: --tls-min-version="" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697241 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697246 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697248 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697251 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697255 2566 flags.go:64] FLAG: --v="2" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697259 2566 flags.go:64] FLAG: --version="false" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697262 2566 flags.go:64] FLAG: --vmodule="" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697266 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.697270 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 03:42:10.028376 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697385 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697389 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697392 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697394 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697397 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697400 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697403 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697405 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697409 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697412 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697414 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697417 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697420 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697422 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697425 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697427 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697431 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697435 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697438 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 03:42:10.032969 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697441 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697444 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697446 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697449 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697451 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697456 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697458 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697461 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697464 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697467 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697470 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697473 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697475 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697477 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697480 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697482 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697485 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697487 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697490 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697493 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 03:42:10.033589 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697495 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697500 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697503 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697505 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697508 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697511 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697514 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697516 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697519 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697521 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697524 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697526 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697528 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697531 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697533 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697536 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697539 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697543 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697545 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697548 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 03:42:10.034183 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697550 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697553 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697556 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697558 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697561 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697564 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697566 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697568 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697571 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697573 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697575 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697578 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697580 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697584 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697587 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697591 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697594 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697597 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697600 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 03:42:10.034796 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697604 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697607 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697610 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697613 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697615 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697618 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697620 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697623 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697625 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697627 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697632 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697634 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697637 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697639 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697642 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697658 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697661 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697664 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697667 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697670 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 03:42:10.035342 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.697672 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 03:42:10.036127 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.698368 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 03:42:10.036127 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.706853 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 09 03:42:10.036127 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.706875 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 03:42:10.036127 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.707678 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 03:42:10.036127 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.707687 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 03:42:10.036127 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707757 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 03:42:10.036127 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707763 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 03:42:10.036127 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707766 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 03:42:10.036127 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707769 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 03:42:10.036127 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707772 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 03:42:10.036127 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707775 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 03:42:10.036127 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707778 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 03:42:10.036127 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707781 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 03:42:10.036127 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707783 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707786 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707789 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707791 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707796 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707799 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707803 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707806 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707809 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707813 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707815 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707823 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707826 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707829 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707832 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707834 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707837 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707841 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707845 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 03:42:10.036538 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707848 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707850 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707853 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707855 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707858 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707862 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707865 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707867 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707871 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707873 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707876 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707879 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707882 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707885 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707888 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707890 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707893 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707896 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707898 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707902 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 03:42:10.229457 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707905 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707908 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707913 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707916 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707919 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707921 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707924 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707927 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707929 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707932 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707935 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707937 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707940 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707942 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707945 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707947 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707950 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707954 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707956 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707959 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 03:42:10.230308 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707962 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707964 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707967 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707969 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707972 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707975 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707978 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707980 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707983 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707985 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707987 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707990 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707994 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.707996 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708000 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708003 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708005 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708008 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708010 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708013 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 03:42:10.231082 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708016 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 03:42:10.231628 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708018 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 03:42:10.231628 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708021 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 03:42:10.231628 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708023 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 03:42:10.231628 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708025 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 03:42:10.231628 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708028 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 03:42:10.231628 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708031 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 03:42:10.231628 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708034 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 03:42:10.231628 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708036 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 03:42:10.231628 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708040 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 03:42:10.231628 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708043 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 03:42:10.231628 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708047 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 03:42:10.231628 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.708053 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 03:42:10.231628 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708188 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 03:42:10.231628 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708194 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708197 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708200 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708202 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708205 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708207 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708210 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708212 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708215 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708218 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708221 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708225 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708229 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708234 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708237 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708240 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708243 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708246 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708249 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 03:42:10.232075 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708252 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708255 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708258 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708260 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708263 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708266 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708269 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708271 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708276 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708279 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708282 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708284 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708287 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708290 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708293 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708295 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708298 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708300 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708303 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708306 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 03:42:10.232580 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708308 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708312 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708316 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708318 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708321 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708325 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708329 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708332 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708334 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708337 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708340 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708342 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708345 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708347 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708350 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708352 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708355 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708358 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708361 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 03:42:10.233154 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708363 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708367 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708369 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708372 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708374 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708378 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708382 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708385 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708388 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708391 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708394 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708396 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708399 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708402 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708405 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708408 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708410 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708413 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708416 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708422 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 03:42:10.233722 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708424 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708427 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708430 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708432 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708435 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708438 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708441 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708443 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708446 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708448 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708451 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708453 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708456 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708460 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708463 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708465 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708468 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708470 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708473 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 03:42:10.234294 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:09.708475 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 03:42:10.234815 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.708481 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 03:42:10.234815 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.708615 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 03:42:10.234815 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.712959 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 03:42:10.234815 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.714571 2566 server.go:1019] "Starting client certificate rotation" Jul 09 03:42:10.234815 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.714676 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 03:42:10.234815 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.714711 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 03:42:10.234815 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.717398 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 03:42:10.234815 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.721904 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 03:42:10.234815 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.737409 2566 log.go:25] "Validated CRI v1 runtime API" Jul 09 03:42:10.234815 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.740131 2566 log.go:25] "Validated CRI v1 image API" Jul 09 03:42:10.234815 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.743469 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 03:42:10.234815 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.746548 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 03:42:10.235234 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.747695 2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d969b0dc-0b27-4b87-823c-3331e587e2f2:/dev/nvme0n1p3 f39a12a7-fd98-4079-81f1-98cc21ddb843:/dev/nvme0n1p4] Jul 09 03:42:10.235234 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.747719 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.752882 2566 manager.go:216] Machine: {Timestamp:2026-07-09 03:42:09.752146205 +0000 UTC m=+0.383806281 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100579 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b589c48fd5a5d2dc60b767d8404d9 SystemUUID:ec2b589c-48fd-5a5d-2dc6-0b767d8404d9 BootID:81288d3d-282d-469c-8b37-b4d942ea2027 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b8:2b:5f:bc:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b8:2b:5f:bc:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:06:a1:4b:85:bd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.752994 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.753080 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.754238 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.754272 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-230.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.755141 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.755152 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.755160 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.755175 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.759362 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.763827 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.763962 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.763992 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.764003 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.764015 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.764026 2566 kubelet.go:398] "Adding apiserver pod source" Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.764035 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 03:42:10.235404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.764386 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.764401 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.766831 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.769335 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.769364 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.770852 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.770865 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.770870 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.770876 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.770885 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.770891 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.770899 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.770905 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.770911 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.770917 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.770927 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.770932 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.771862 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.771872 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.775828 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-230.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.775833 2566 server.go:1268] "Started kubelet" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.775867 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.775915 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.776668 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.776673 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.776884 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.777460 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.778010 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.781286 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.780732 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-230.ec2.internal.18c08220c9da427d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-230.ec2.internal,UID:ip-10-0-137-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-230.ec2.internal,},FirstTimestamp:2026-07-09 03:42:09.775805053 +0000 UTC m=+0.407465176,LastTimestamp:2026-07-09 03:42:09.775805053 +0000 UTC m=+0.407465176,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-230.ec2.internal,}" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.785059 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.785070 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.785124 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.785799 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.785960 2566 factory.go:55] Registering systemd factory Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.785979 2566 factory.go:223] Registration of the systemd container factory successfully Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.785965 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.786032 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.786121 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.786131 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.786229 2566 factory.go:153] Registering CRI-O factory Jul 09 03:42:10.236259 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.786240 2566 factory.go:223] Registration of the crio container factory successfully Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.786302 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.786327 2566 factory.go:103] Registering Raw factory Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.786342 2566 manager.go:1196] Started watching for new ooms in manager Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.786659 2566 manager.go:349] Starting recovery of all containers Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.794171 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dbcbf" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.799013 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.799133 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.799291 2566 manager.go:354] Recovery completed Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.799797 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dbcbf" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.800312 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.804407 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.806761 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasSufficientMemory" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.806796 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.806811 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasSufficientPID" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.807734 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.807746 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.807760 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.807821 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.808726 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasSufficientMemory" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.808753 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.808767 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasSufficientPID" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.808871 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-230.ec2.internal.18c08220cbb2dec2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-230.ec2.internal,UID:ip-10-0-137-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-230.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-230.ec2.internal,},FirstTimestamp:2026-07-09 03:42:09.80677805 +0000 UTC m=+0.438438129,LastTimestamp:2026-07-09 03:42:09.80677805 +0000 UTC m=+0.438438129,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-230.ec2.internal,}" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.809847 2566 policy_none.go:49] "None policy: Start" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.809860 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.809870 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.811818 2566 policy_none.go:47] "Start" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.857963 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.858004 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.858028 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.858060 2566 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.860313 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.868315 2566 manager.go:337] "Starting Device Plugin manager" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.868348 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.868359 2566 server.go:87] "Starting device plugin registration server" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.868600 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.868610 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.868706 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.868786 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 03:42:10.237423 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.868795 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.869310 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.869361 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.959020 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-230.ec2.internal"] Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.959104 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.961023 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasSufficientMemory" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.961056 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.961066 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasSufficientPID" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.963339 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.963448 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.963481 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.964441 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasSufficientMemory" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.964469 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.964479 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasSufficientPID" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.964441 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasSufficientMemory" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.964537 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.964551 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasSufficientPID" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.966803 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.966835 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.967685 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasSufficientMemory" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.967716 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.967728 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasSufficientPID" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.969175 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.969915 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasSufficientMemory" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.969941 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.969951 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeHasSufficientPID" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.969970 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.979067 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.979088 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-230.ec2.internal\": node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.983406 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-230.ec2.internal\" not found" node="ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.987077 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e59dcb1db041284efb34c27cebcb17d3-config\") pod \"kube-apiserver-proxy-ip-10-0-137-230.ec2.internal\" (UID: \"e59dcb1db041284efb34c27cebcb17d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.987114 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f94e666b22f673976f4d41923aab2d9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal\" (UID: \"2f94e666b22f673976f4d41923aab2d9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:09.987138 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f94e666b22f673976f4d41923aab2d9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal\" (UID: \"2f94e666b22f673976f4d41923aab2d9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.439083 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:09.987880 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-230.ec2.internal\" not found" node="ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.441492 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:10.007310 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:10.441492 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:10.087990 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e59dcb1db041284efb34c27cebcb17d3-config\") pod \"kube-apiserver-proxy-ip-10-0-137-230.ec2.internal\" (UID: \"e59dcb1db041284efb34c27cebcb17d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.441492 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:10.088022 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f94e666b22f673976f4d41923aab2d9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal\" (UID: \"2f94e666b22f673976f4d41923aab2d9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.441492 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:10.088048 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f94e666b22f673976f4d41923aab2d9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal\" (UID: \"2f94e666b22f673976f4d41923aab2d9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.441492 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:10.088083 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f94e666b22f673976f4d41923aab2d9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal\" (UID: \"2f94e666b22f673976f4d41923aab2d9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.441492 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:10.088098 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f94e666b22f673976f4d41923aab2d9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal\" (UID: \"2f94e666b22f673976f4d41923aab2d9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.441492 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:10.088083 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e59dcb1db041284efb34c27cebcb17d3-config\") pod \"kube-apiserver-proxy-ip-10-0-137-230.ec2.internal\" (UID: \"e59dcb1db041284efb34c27cebcb17d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.441492 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:10.108131 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:10.441492 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:10.208665 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:10.441492 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:10.287227 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.441492 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:10.291362 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-230.ec2.internal" Jul 09 03:42:10.441492 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:10.309414 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:10.441492 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:10.410027 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:10.650514 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:10.510511 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:10.650514 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:10.611118 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:10.838963 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:10.711671 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:10.838963 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:10.713824 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 03:42:10.838963 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:10.713979 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 03:42:10.838963 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:10.805597 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 03:37:10 +0000 UTC" deadline="2028-04-11 23:43:49.838007396 +0000 UTC" Jul 09 03:42:10.838963 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:10.805625 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15428h1m39.032385432s" Jul 09 03:42:10.838963 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:10.812104 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:11.060563 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:10.912697 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:11.060563 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:11.012835 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:11.247131 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:11.113748 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:11.247131 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:11.177510 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f94e666b22f673976f4d41923aab2d9.slice/crio-88e90a00b3648e6be840bdd3452c34f0c39608c84e5022cdf63904b3eab5eab9 WatchSource:0}: Error finding container 88e90a00b3648e6be840bdd3452c34f0c39608c84e5022cdf63904b3eab5eab9: Status 404 returned error can't find the container with id 88e90a00b3648e6be840bdd3452c34f0c39608c84e5022cdf63904b3eab5eab9 Jul 09 03:42:11.247131 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:11.177959 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode59dcb1db041284efb34c27cebcb17d3.slice/crio-b235a686ecfac05d9d465d1c7a74777c4ca9806bcb45a2b90baf05bdccc5e016 WatchSource:0}: Error finding container b235a686ecfac05d9d465d1c7a74777c4ca9806bcb45a2b90baf05bdccc5e016: Status 404 returned error can't find the container with id b235a686ecfac05d9d465d1c7a74777c4ca9806bcb45a2b90baf05bdccc5e016 Jul 09 03:42:11.247131 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.187235 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 03:42:11.247131 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:11.214187 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:11.247131 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.214836 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 03:42:11.485772 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:11.315390 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-230.ec2.internal\" not found" Jul 09 03:42:11.485772 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.322887 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 03:42:11.485772 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.334260 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 03:42:11.485772 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.387591 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal" Jul 09 03:42:11.485772 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.401829 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 03:42:11.485772 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.403056 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-230.ec2.internal" Jul 09 03:42:11.485772 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.411472 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 03:42:11.694971 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.581792 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.766255 2566 apiserver.go:52] "Watching apiserver" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.774781 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.775175 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-h5d9p","kube-system/kube-apiserver-proxy-ip-10-0-137-230.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v","openshift-cluster-node-tuning-operator/tuned-gqxlk","openshift-dns/node-resolver-mf9s5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal","openshift-multus/multus-additional-cni-plugins-tkpv2","openshift-multus/network-metrics-daemon-r9c5k","openshift-ovn-kubernetes/ovnkube-node-s5kh8","kube-system/global-pull-secret-syncer-k9f2k","kube-system/konnectivity-agent-nfr48","openshift-image-registry/node-ca-6dggs","openshift-multus/multus-nk6tf","openshift-network-diagnostics/network-check-target-vkw66"] Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.779398 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.782720 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.784341 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.786245 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.786974 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.789419 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mf9s5" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.792278 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:11.792325 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.795118 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6dggs" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799130 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nfr48" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799070 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:11.799259 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799480 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799507 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-system-cni-dir\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799531 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-os-release\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799568 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/167f4240-06e8-45a9-8770-56c395498f7d-cni-binary-copy\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799591 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-multus-socket-dir-parent\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799614 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-run-k8s-cni-cncf-io\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.832665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799637 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-var-lib-cni-multus\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799660 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-run-multus-certs\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799685 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-cni-binary-copy\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799706 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-var-lib-cni-bin\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799745 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znnpf\" (UniqueName: \"kubernetes.io/projected/167f4240-06e8-45a9-8770-56c395498f7d-kube-api-access-znnpf\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799769 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-socket-dir\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799792 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-etc-selinux\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799815 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-sys-fs\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4hrz\" (UniqueName: \"kubernetes.io/projected/e894676b-96c1-48de-b161-d9a92e86fcf6-kube-api-access-b4hrz\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799912 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-kubernetes\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799937 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-cnibin\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.799964 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800011 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-cnibin\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800034 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-sysctl-d\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800051 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-lib-modules\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800070 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-var-lib-kubelet\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.834764 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800086 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-system-cni-dir\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800105 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq524\" (UniqueName: \"kubernetes.io/projected/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-kube-api-access-kq524\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800122 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-multus-conf-dir\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800137 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-device-dir\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800153 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-sysconfig\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800177 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-systemd\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800206 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-sys\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800227 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c205b758-9f45-4a64-be35-efa418109da9-etc-tuned\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800261 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-multus-cni-dir\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800282 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-run-netns\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800308 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-var-lib-kubelet\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800330 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800351 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-etc-kubernetes\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800373 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800402 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-modprobe-d\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800423 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-sysctl-conf\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800443 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-run\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.835480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800463 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-host\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800485 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-os-release\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800510 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-hostroot\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800533 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/167f4240-06e8-45a9-8770-56c395498f7d-multus-daemon-config\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800579 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-registration-dir\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800605 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqbfp\" (UniqueName: \"kubernetes.io/projected/c205b758-9f45-4a64-be35-efa418109da9-kube-api-access-vqbfp\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.800650 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c205b758-9f45-4a64-be35-efa418109da9-tmp\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.805485 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.805550 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.806019 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h5d9p" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.807117 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.807168 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9fhjl\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.807540 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.807826 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g2r4j\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.807997 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.808109 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.808181 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.808522 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.808533 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.808791 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.808994 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vvndz\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.809371 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cf7vs\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.809428 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.809652 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7vmwv\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.809822 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.810008 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.810511 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.810566 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.810732 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.810772 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 03:42:11.836219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.810831 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dvd9f\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.810891 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.811047 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.811170 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.811196 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.811239 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.811257 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2htfn\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.811385 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-d5nxv\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.813194 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:11.813259 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.818969 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.819002 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.818975 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.819083 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-drbm9\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.819096 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.819160 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.819204 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.819360 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 03:42:11.837456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.824745 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hdr4w" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.860410 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hdr4w" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.863443 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-230.ec2.internal" event={"ID":"e59dcb1db041284efb34c27cebcb17d3","Type":"ContainerStarted","Data":"b235a686ecfac05d9d465d1c7a74777c4ca9806bcb45a2b90baf05bdccc5e016"} Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.864401 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal" event={"ID":"2f94e666b22f673976f4d41923aab2d9","Type":"ContainerStarted","Data":"88e90a00b3648e6be840bdd3452c34f0c39608c84e5022cdf63904b3eab5eab9"} Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.887346 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901554 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6vp6\" (UniqueName: \"kubernetes.io/projected/6da3145c-5f91-44ca-8465-ce298a4ee12e-kube-api-access-c6vp6\") pod \"node-ca-6dggs\" (UID: \"6da3145c-5f91-44ca-8465-ce298a4ee12e\") " pod="openshift-image-registry/node-ca-6dggs" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901596 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-system-cni-dir\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901619 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-os-release\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901636 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/167f4240-06e8-45a9-8770-56c395498f7d-cni-binary-copy\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901698 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-var-lib-cni-multus\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901735 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-sys-fs\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901748 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-system-cni-dir\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901763 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-kubernetes\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901824 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-os-release\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901858 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c205b758-9f45-4a64-be35-efa418109da9-etc-tuned\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901888 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-cni-binary-copy\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901911 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-var-lib-cni-bin\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901935 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/69b8b6fa-1fda-4c62-81d1-c9b27231a274-dbus\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901979 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-var-lib-cni-bin\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.957440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.901999 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03b1ba2d-7000-46e5-af08-6977005ca09d-env-overrides\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902037 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-cnibin\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902056 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-var-lib-kubelet\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902073 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdnlp\" (UniqueName: \"kubernetes.io/projected/758f53c8-f537-4587-9523-3c9ccedafd4a-kube-api-access-kdnlp\") pod \"iptables-alerter-h5d9p\" (UID: \"758f53c8-f537-4587-9523-3c9ccedafd4a\") " pod="openshift-network-operator/iptables-alerter-h5d9p" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902125 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-kubelet\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902226 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-run-systemd\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902259 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsbcc\" (UniqueName: \"kubernetes.io/projected/03b1ba2d-7000-46e5-af08-6977005ca09d-kube-api-access-nsbcc\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902289 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-system-cni-dir\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902313 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq524\" (UniqueName: \"kubernetes.io/projected/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-kube-api-access-kq524\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902334 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-multus-conf-dir\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902350 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-sysconfig\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902365 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-systemd\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902393 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902422 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/167f4240-06e8-45a9-8770-56c395498f7d-cni-binary-copy\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902436 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-systemd-units\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902473 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-cni-netd\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902521 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-system-cni-dir\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.958779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902528 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-kubernetes\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902579 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/69b8b6fa-1fda-4c62-81d1-c9b27231a274-kubelet-config\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902586 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-cni-binary-copy\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902625 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03b1ba2d-7000-46e5-af08-6977005ca09d-ovn-node-metrics-cert\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902643 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-etc-kubernetes\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902660 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-sys-fs\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902673 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-log-socket\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902744 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-systemd\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902809 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-etc-kubernetes\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902817 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-sysconfig\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902857 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-os-release\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902879 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/167f4240-06e8-45a9-8770-56c395498f7d-multus-daemon-config\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902880 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-var-lib-kubelet\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902907 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-cnibin\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902917 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-multus-conf-dir\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902967 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c205b758-9f45-4a64-be35-efa418109da9-tmp\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902987 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-var-lib-cni-multus\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.959378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.902993 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-run-netns\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.903027 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqbfp\" (UniqueName: \"kubernetes.io/projected/c205b758-9f45-4a64-be35-efa418109da9-kube-api-access-vqbfp\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.903045 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-os-release\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.903054 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.903082 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6da3145c-5f91-44ca-8465-ce298a4ee12e-host\") pod \"node-ca-6dggs\" (UID: \"6da3145c-5f91-44ca-8465-ce298a4ee12e\") " pod="openshift-image-registry/node-ca-6dggs" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.903114 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.903136 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-multus-socket-dir-parent\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.903156 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-run-k8s-cni-cncf-io\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.903213 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-run-multus-certs\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.903253 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-socket-dir\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.903292 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4hrz\" (UniqueName: \"kubernetes.io/projected/e894676b-96c1-48de-b161-d9a92e86fcf6-kube-api-access-b4hrz\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.903864 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znnpf\" (UniqueName: \"kubernetes.io/projected/167f4240-06e8-45a9-8770-56c395498f7d-kube-api-access-znnpf\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.903911 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-etc-selinux\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.903948 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/167f4240-06e8-45a9-8770-56c395498f7d-multus-daemon-config\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904062 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-lib-modules\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904145 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-run-multus-certs\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904219 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-run-k8s-cni-cncf-io\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.960024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.903940 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-lib-modules\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904344 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-multus-socket-dir-parent\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904345 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-socket-dir\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904382 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-sys\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904415 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/758f53c8-f537-4587-9523-3c9ccedafd4a-host-slash\") pod \"iptables-alerter-h5d9p\" (UID: \"758f53c8-f537-4587-9523-3c9ccedafd4a\") " pod="openshift-network-operator/iptables-alerter-h5d9p" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904439 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-run-openvswitch\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904468 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904483 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-cni-bin\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904511 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03b1ba2d-7000-46e5-af08-6977005ca09d-ovnkube-config\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904545 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-sys\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904561 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-cnibin\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904591 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904618 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-sysctl-d\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904656 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-etc-openvswitch\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904681 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-node-log\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-run-ovn-kubernetes\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904749 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6ac5e518-8402-4297-8ce7-4f083d9c2df2-hosts-file\") pod \"node-resolver-mf9s5\" (UID: \"6ac5e518-8402-4297-8ce7-4f083d9c2df2\") " pod="openshift-dns/node-resolver-mf9s5" Jul 09 03:42:11.960617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904687 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-etc-selinux\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904775 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/175280b2-1ecd-4cc1-a96a-5a9a13207aef-agent-certs\") pod \"konnectivity-agent-nfr48\" (UID: \"175280b2-1ecd-4cc1-a96a-5a9a13207aef\") " pod="kube-system/konnectivity-agent-nfr48" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904778 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-cnibin\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904810 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-sysctl-d\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904821 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-device-dir\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904863 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-slash\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904891 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dd75\" (UniqueName: \"kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75\") pod \"network-check-target-vkw66\" (UID: \"ecce4016-016f-4060-8fcc-d350522156ad\") " pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904922 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnhfd\" (UniqueName: \"kubernetes.io/projected/6ac5e518-8402-4297-8ce7-4f083d9c2df2-kube-api-access-gnhfd\") pod \"node-resolver-mf9s5\" (UID: \"6ac5e518-8402-4297-8ce7-4f083d9c2df2\") " pod="openshift-dns/node-resolver-mf9s5" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904939 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-device-dir\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904959 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.904963 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-multus-cni-dir\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905000 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-run-netns\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905036 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-var-lib-kubelet\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905061 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-run-netns\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905065 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-multus-cni-dir\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905096 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905107 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-host-var-lib-kubelet\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.961285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905146 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ac5e518-8402-4297-8ce7-4f083d9c2df2-tmp-dir\") pod \"node-resolver-mf9s5\" (UID: \"6ac5e518-8402-4297-8ce7-4f083d9c2df2\") " pod="openshift-dns/node-resolver-mf9s5" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905176 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905201 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905253 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-modprobe-d\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905277 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-sysctl-conf\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905311 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-run\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905333 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-host\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905360 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nv57\" (UniqueName: \"kubernetes.io/projected/c275241c-186f-492a-952a-cb6ed44c560f-kube-api-access-9nv57\") pod \"network-metrics-daemon-r9c5k\" (UID: \"c275241c-186f-492a-952a-cb6ed44c560f\") " pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905384 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/758f53c8-f537-4587-9523-3c9ccedafd4a-iptables-alerter-script\") pod \"iptables-alerter-h5d9p\" (UID: \"758f53c8-f537-4587-9523-3c9ccedafd4a\") " pod="openshift-network-operator/iptables-alerter-h5d9p" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905413 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-hostroot\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905437 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-registration-dir\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905462 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/175280b2-1ecd-4cc1-a96a-5a9a13207aef-konnectivity-ca\") pod \"konnectivity-agent-nfr48\" (UID: \"175280b2-1ecd-4cc1-a96a-5a9a13207aef\") " pod="kube-system/konnectivity-agent-nfr48" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905486 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-var-lib-openvswitch\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905516 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03b1ba2d-7000-46e5-af08-6977005ca09d-ovnkube-script-lib\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905524 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905539 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs\") pod \"network-metrics-daemon-r9c5k\" (UID: \"c275241c-186f-492a-952a-cb6ed44c560f\") " pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905570 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6da3145c-5f91-44ca-8465-ce298a4ee12e-serviceca\") pod \"node-ca-6dggs\" (UID: \"6da3145c-5f91-44ca-8465-ce298a4ee12e\") " pod="openshift-image-registry/node-ca-6dggs" Jul 09 03:42:11.961899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905576 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.962545 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905595 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-run-ovn\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:11.962545 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905649 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-sysctl-conf\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.962545 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905680 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-run\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.962545 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905718 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/167f4240-06e8-45a9-8770-56c395498f7d-hostroot\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.962545 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905733 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-host\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.962545 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905778 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e894676b-96c1-48de-b161-d9a92e86fcf6-registration-dir\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:11.962545 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.905810 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c205b758-9f45-4a64-be35-efa418109da9-etc-modprobe-d\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.962545 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.906572 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c205b758-9f45-4a64-be35-efa418109da9-tmp\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.962545 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.906621 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c205b758-9f45-4a64-be35-efa418109da9-etc-tuned\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.962545 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.915515 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znnpf\" (UniqueName: \"kubernetes.io/projected/167f4240-06e8-45a9-8770-56c395498f7d-kube-api-access-znnpf\") pod \"multus-nk6tf\" (UID: \"167f4240-06e8-45a9-8770-56c395498f7d\") " pod="openshift-multus/multus-nk6tf" Jul 09 03:42:11.962545 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.915537 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq524\" (UniqueName: \"kubernetes.io/projected/e0abbca0-fa5c-40f8-be6d-225daabb1d4e-kube-api-access-kq524\") pod \"multus-additional-cni-plugins-tkpv2\" (UID: \"e0abbca0-fa5c-40f8-be6d-225daabb1d4e\") " pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:11.962545 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.915538 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqbfp\" (UniqueName: \"kubernetes.io/projected/c205b758-9f45-4a64-be35-efa418109da9-kube-api-access-vqbfp\") pod \"tuned-gqxlk\" (UID: \"c205b758-9f45-4a64-be35-efa418109da9\") " pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:11.962545 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:11.917309 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4hrz\" (UniqueName: \"kubernetes.io/projected/e894676b-96c1-48de-b161-d9a92e86fcf6-kube-api-access-b4hrz\") pod \"aws-ebs-csi-driver-node-8z96v\" (UID: \"e894676b-96c1-48de-b161-d9a92e86fcf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006535 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-log-socket\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006570 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-run-netns\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006588 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6da3145c-5f91-44ca-8465-ce298a4ee12e-host\") pod \"node-ca-6dggs\" (UID: \"6da3145c-5f91-44ca-8465-ce298a4ee12e\") " pod="openshift-image-registry/node-ca-6dggs" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006609 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/758f53c8-f537-4587-9523-3c9ccedafd4a-host-slash\") pod \"iptables-alerter-h5d9p\" (UID: \"758f53c8-f537-4587-9523-3c9ccedafd4a\") " pod="openshift-network-operator/iptables-alerter-h5d9p" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006629 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-run-openvswitch\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006699 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-run-openvswitch\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006700 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-run-netns\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006699 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6da3145c-5f91-44ca-8465-ce298a4ee12e-host\") pod \"node-ca-6dggs\" (UID: \"6da3145c-5f91-44ca-8465-ce298a4ee12e\") " pod="openshift-image-registry/node-ca-6dggs" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006748 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/758f53c8-f537-4587-9523-3c9ccedafd4a-host-slash\") pod \"iptables-alerter-h5d9p\" (UID: \"758f53c8-f537-4587-9523-3c9ccedafd4a\") " pod="openshift-network-operator/iptables-alerter-h5d9p" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006775 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-cni-bin\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006806 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-cni-bin\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006835 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-log-socket\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006881 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03b1ba2d-7000-46e5-af08-6977005ca09d-ovnkube-config\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006904 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-etc-openvswitch\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006927 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-node-log\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006947 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-run-ovn-kubernetes\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006964 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6ac5e518-8402-4297-8ce7-4f083d9c2df2-hosts-file\") pod \"node-resolver-mf9s5\" (UID: \"6ac5e518-8402-4297-8ce7-4f083d9c2df2\") " pod="openshift-dns/node-resolver-mf9s5" Jul 09 03:42:12.092770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006971 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-etc-openvswitch\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006980 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/175280b2-1ecd-4cc1-a96a-5a9a13207aef-agent-certs\") pod \"konnectivity-agent-nfr48\" (UID: \"175280b2-1ecd-4cc1-a96a-5a9a13207aef\") " pod="kube-system/konnectivity-agent-nfr48" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.006999 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-node-log\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007001 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-slash\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007034 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-slash\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007032 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-run-ovn-kubernetes\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007035 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6ac5e518-8402-4297-8ce7-4f083d9c2df2-hosts-file\") pod \"node-resolver-mf9s5\" (UID: \"6ac5e518-8402-4297-8ce7-4f083d9c2df2\") " pod="openshift-dns/node-resolver-mf9s5" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007060 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dd75\" (UniqueName: \"kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75\") pod \"network-check-target-vkw66\" (UID: \"ecce4016-016f-4060-8fcc-d350522156ad\") " pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007083 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnhfd\" (UniqueName: \"kubernetes.io/projected/6ac5e518-8402-4297-8ce7-4f083d9c2df2-kube-api-access-gnhfd\") pod \"node-resolver-mf9s5\" (UID: \"6ac5e518-8402-4297-8ce7-4f083d9c2df2\") " pod="openshift-dns/node-resolver-mf9s5" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007117 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007141 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ac5e518-8402-4297-8ce7-4f083d9c2df2-tmp-dir\") pod \"node-resolver-mf9s5\" (UID: \"6ac5e518-8402-4297-8ce7-4f083d9c2df2\") " pod="openshift-dns/node-resolver-mf9s5" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007181 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007225 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nv57\" (UniqueName: \"kubernetes.io/projected/c275241c-186f-492a-952a-cb6ed44c560f-kube-api-access-9nv57\") pod \"network-metrics-daemon-r9c5k\" (UID: \"c275241c-186f-492a-952a-cb6ed44c560f\") " pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007322 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/758f53c8-f537-4587-9523-3c9ccedafd4a-iptables-alerter-script\") pod \"iptables-alerter-h5d9p\" (UID: \"758f53c8-f537-4587-9523-3c9ccedafd4a\") " pod="openshift-network-operator/iptables-alerter-h5d9p" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007351 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/175280b2-1ecd-4cc1-a96a-5a9a13207aef-konnectivity-ca\") pod \"konnectivity-agent-nfr48\" (UID: \"175280b2-1ecd-4cc1-a96a-5a9a13207aef\") " pod="kube-system/konnectivity-agent-nfr48" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007379 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-var-lib-openvswitch\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007405 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03b1ba2d-7000-46e5-af08-6977005ca09d-ovnkube-script-lib\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.094405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007422 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ac5e518-8402-4297-8ce7-4f083d9c2df2-tmp-dir\") pod \"node-resolver-mf9s5\" (UID: \"6ac5e518-8402-4297-8ce7-4f083d9c2df2\") " pod="openshift-dns/node-resolver-mf9s5" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007438 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs\") pod \"network-metrics-daemon-r9c5k\" (UID: \"c275241c-186f-492a-952a-cb6ed44c560f\") " pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007463 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6da3145c-5f91-44ca-8465-ce298a4ee12e-serviceca\") pod \"node-ca-6dggs\" (UID: \"6da3145c-5f91-44ca-8465-ce298a4ee12e\") " pod="openshift-image-registry/node-ca-6dggs" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007474 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-var-lib-openvswitch\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007488 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-run-ovn\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007524 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6vp6\" (UniqueName: \"kubernetes.io/projected/6da3145c-5f91-44ca-8465-ce298a4ee12e-kube-api-access-c6vp6\") pod \"node-ca-6dggs\" (UID: \"6da3145c-5f91-44ca-8465-ce298a4ee12e\") " pod="openshift-image-registry/node-ca-6dggs" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007533 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-run-ovn\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007574 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/69b8b6fa-1fda-4c62-81d1-c9b27231a274-dbus\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007598 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03b1ba2d-7000-46e5-af08-6977005ca09d-env-overrides\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007608 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03b1ba2d-7000-46e5-af08-6977005ca09d-ovnkube-config\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007624 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdnlp\" (UniqueName: \"kubernetes.io/projected/758f53c8-f537-4587-9523-3c9ccedafd4a-kube-api-access-kdnlp\") pod \"iptables-alerter-h5d9p\" (UID: \"758f53c8-f537-4587-9523-3c9ccedafd4a\") " pod="openshift-network-operator/iptables-alerter-h5d9p" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007646 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-kubelet\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007662 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-run-systemd\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007682 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsbcc\" (UniqueName: \"kubernetes.io/projected/03b1ba2d-7000-46e5-af08-6977005ca09d-kube-api-access-nsbcc\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007707 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007738 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-systemd-units\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007755 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-cni-netd\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.095549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007782 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/69b8b6fa-1fda-4c62-81d1-c9b27231a274-kubelet-config\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007798 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03b1ba2d-7000-46e5-af08-6977005ca09d-ovn-node-metrics-cert\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.007849 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.007933 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs podName:c275241c-186f-492a-952a-cb6ed44c560f nodeName:}" failed. No retries permitted until 2026-07-09 03:42:12.507910148 +0000 UTC m=+3.139570242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs") pod "network-metrics-daemon-r9c5k" (UID: "c275241c-186f-492a-952a-cb6ed44c560f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.007989 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/175280b2-1ecd-4cc1-a96a-5a9a13207aef-konnectivity-ca\") pod \"konnectivity-agent-nfr48\" (UID: \"175280b2-1ecd-4cc1-a96a-5a9a13207aef\") " pod="kube-system/konnectivity-agent-nfr48" Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.008029 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03b1ba2d-7000-46e5-af08-6977005ca09d-ovnkube-script-lib\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.008058 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-run-systemd\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.008125 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6da3145c-5f91-44ca-8465-ce298a4ee12e-serviceca\") pod \"node-ca-6dggs\" (UID: \"6da3145c-5f91-44ca-8465-ce298a4ee12e\") " pod="openshift-image-registry/node-ca-6dggs" Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.008169 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/69b8b6fa-1fda-4c62-81d1-c9b27231a274-dbus\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.008186 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-kubelet\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.008226 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-systemd-units\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.008239 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03b1ba2d-7000-46e5-af08-6977005ca09d-host-cni-netd\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.008278 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/69b8b6fa-1fda-4c62-81d1-c9b27231a274-kubelet-config\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.008318 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.008382 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret podName:69b8b6fa-1fda-4c62-81d1-c9b27231a274 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:12.508364033 +0000 UTC m=+3.140024105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret") pod "global-pull-secret-syncer-k9f2k" (UID: "69b8b6fa-1fda-4c62-81d1-c9b27231a274") : object "kube-system"/"original-pull-secret" not registered Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.008389 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03b1ba2d-7000-46e5-af08-6977005ca09d-env-overrides\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.008687 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/758f53c8-f537-4587-9523-3c9ccedafd4a-iptables-alerter-script\") pod \"iptables-alerter-h5d9p\" (UID: \"758f53c8-f537-4587-9523-3c9ccedafd4a\") " pod="openshift-network-operator/iptables-alerter-h5d9p" Jul 09 03:42:12.097470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.010104 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/175280b2-1ecd-4cc1-a96a-5a9a13207aef-agent-certs\") pod \"konnectivity-agent-nfr48\" (UID: \"175280b2-1ecd-4cc1-a96a-5a9a13207aef\") " pod="kube-system/konnectivity-agent-nfr48" Jul 09 03:42:12.098899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.010645 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03b1ba2d-7000-46e5-af08-6977005ca09d-ovn-node-metrics-cert\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.098899 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.014365 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 03:42:12.098899 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.014387 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 03:42:12.098899 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.014398 2566 projected.go:196] Error preparing data for projected volume kube-api-access-4dd75 for pod openshift-network-diagnostics/network-check-target-vkw66: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:42:12.098899 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.014465 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75 podName:ecce4016-016f-4060-8fcc-d350522156ad nodeName:}" failed. No retries permitted until 2026-07-09 03:42:12.514446037 +0000 UTC m=+3.146106124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4dd75" (UniqueName: "kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75") pod "network-check-target-vkw66" (UID: "ecce4016-016f-4060-8fcc-d350522156ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:42:12.098899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.017696 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnhfd\" (UniqueName: \"kubernetes.io/projected/6ac5e518-8402-4297-8ce7-4f083d9c2df2-kube-api-access-gnhfd\") pod \"node-resolver-mf9s5\" (UID: \"6ac5e518-8402-4297-8ce7-4f083d9c2df2\") " pod="openshift-dns/node-resolver-mf9s5" Jul 09 03:42:12.098899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.017975 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6vp6\" (UniqueName: \"kubernetes.io/projected/6da3145c-5f91-44ca-8465-ce298a4ee12e-kube-api-access-c6vp6\") pod \"node-ca-6dggs\" (UID: \"6da3145c-5f91-44ca-8465-ce298a4ee12e\") " pod="openshift-image-registry/node-ca-6dggs" Jul 09 03:42:12.098899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.018279 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nv57\" (UniqueName: \"kubernetes.io/projected/c275241c-186f-492a-952a-cb6ed44c560f-kube-api-access-9nv57\") pod \"network-metrics-daemon-r9c5k\" (UID: \"c275241c-186f-492a-952a-cb6ed44c560f\") " pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:12.098899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.019569 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsbcc\" (UniqueName: \"kubernetes.io/projected/03b1ba2d-7000-46e5-af08-6977005ca09d-kube-api-access-nsbcc\") pod \"ovnkube-node-s5kh8\" (UID: \"03b1ba2d-7000-46e5-af08-6977005ca09d\") " pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.098899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.019608 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdnlp\" (UniqueName: \"kubernetes.io/projected/758f53c8-f537-4587-9523-3c9ccedafd4a-kube-api-access-kdnlp\") pod \"iptables-alerter-h5d9p\" (UID: \"758f53c8-f537-4587-9523-3c9ccedafd4a\") " pod="openshift-network-operator/iptables-alerter-h5d9p" Jul 09 03:42:12.098899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.092372 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tkpv2" Jul 09 03:42:12.106247 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.105730 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nk6tf" Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.113286 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.119564 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.123631 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mf9s5" Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.129739 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6dggs" Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.137111 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nfr48" Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:12.140100 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0abbca0_fa5c_40f8_be6d_225daabb1d4e.slice/crio-e389b716b77267daf91c91f63b37bb1afca09ff5b0ede2e40809bcd2cc60d42c WatchSource:0}: Error finding container e389b716b77267daf91c91f63b37bb1afca09ff5b0ede2e40809bcd2cc60d42c: Status 404 returned error can't find the container with id e389b716b77267daf91c91f63b37bb1afca09ff5b0ede2e40809bcd2cc60d42c Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:12.142702 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod167f4240_06e8_45a9_8770_56c395498f7d.slice/crio-0f274a147b1756da2335f72bc40f448cb7d935f6d7168080f05b6aeeac4be5d7 WatchSource:0}: Error finding container 0f274a147b1756da2335f72bc40f448cb7d935f6d7168080f05b6aeeac4be5d7: Status 404 returned error can't find the container with id 0f274a147b1756da2335f72bc40f448cb7d935f6d7168080f05b6aeeac4be5d7 Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.150294 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h5d9p" Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.156480 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:12.173854 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode894676b_96c1_48de_b161_d9a92e86fcf6.slice/crio-b766d3c5b40e961180981809ce6e9ea67406d702e5694a251f2858538b7c07b6 WatchSource:0}: Error finding container b766d3c5b40e961180981809ce6e9ea67406d702e5694a251f2858538b7c07b6: Status 404 returned error can't find the container with id b766d3c5b40e961180981809ce6e9ea67406d702e5694a251f2858538b7c07b6 Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:12.175734 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ac5e518_8402_4297_8ce7_4f083d9c2df2.slice/crio-95ebb0e3dc16e7e34f6b4cde89ab9cd6713ee9770bc92dbf9ec039275f4a6699 WatchSource:0}: Error finding container 95ebb0e3dc16e7e34f6b4cde89ab9cd6713ee9770bc92dbf9ec039275f4a6699: Status 404 returned error can't find the container with id 95ebb0e3dc16e7e34f6b4cde89ab9cd6713ee9770bc92dbf9ec039275f4a6699 Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:12.181022 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc205b758_9f45_4a64_be35_efa418109da9.slice/crio-84ec6e9885832c16dc49ad7316d3f69bdf97bda063e8996b9a22a498baa9c6b1 WatchSource:0}: Error finding container 84ec6e9885832c16dc49ad7316d3f69bdf97bda063e8996b9a22a498baa9c6b1: Status 404 returned error can't find the container with id 84ec6e9885832c16dc49ad7316d3f69bdf97bda063e8996b9a22a498baa9c6b1 Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:12.183433 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da3145c_5f91_44ca_8465_ce298a4ee12e.slice/crio-27503fa3fa4d8a97b3f4c79d93226b8176e67cdb179247471899ab10e48fc108 WatchSource:0}: Error finding container 27503fa3fa4d8a97b3f4c79d93226b8176e67cdb179247471899ab10e48fc108: Status 404 returned error can't find the container with id 27503fa3fa4d8a97b3f4c79d93226b8176e67cdb179247471899ab10e48fc108 Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:12.192219 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod175280b2_1ecd_4cc1_a96a_5a9a13207aef.slice/crio-d9104dfc27f242459781c4e924adb13c21883703b5c02f413c20d96c6a074942 WatchSource:0}: Error finding container d9104dfc27f242459781c4e924adb13c21883703b5c02f413c20d96c6a074942: Status 404 returned error can't find the container with id d9104dfc27f242459781c4e924adb13c21883703b5c02f413c20d96c6a074942 Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:12.193173 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod758f53c8_f537_4587_9523_3c9ccedafd4a.slice/crio-f9a865d3a6bd80a113c143153e680216e8cf47a56cdf0a909a962ba35403f458 WatchSource:0}: Error finding container f9a865d3a6bd80a113c143153e680216e8cf47a56cdf0a909a962ba35403f458: Status 404 returned error can't find the container with id f9a865d3a6bd80a113c143153e680216e8cf47a56cdf0a909a962ba35403f458 Jul 09 03:42:12.200786 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:12.197341 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b1ba2d_7000_46e5_af08_6977005ca09d.slice/crio-3ac146a19ceaf6e8444c3ef57f5fb80595367dfe8c072c7158cc240c9864c85e WatchSource:0}: Error finding container 3ac146a19ceaf6e8444c3ef57f5fb80595367dfe8c072c7158cc240c9864c85e: Status 404 returned error can't find the container with id 3ac146a19ceaf6e8444c3ef57f5fb80595367dfe8c072c7158cc240c9864c85e Jul 09 03:42:12.550114 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.518934 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dd75\" (UniqueName: \"kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75\") pod \"network-check-target-vkw66\" (UID: \"ecce4016-016f-4060-8fcc-d350522156ad\") " pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:12.550114 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.518989 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs\") pod \"network-metrics-daemon-r9c5k\" (UID: \"c275241c-186f-492a-952a-cb6ed44c560f\") " pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:12.550114 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.519029 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:12.550114 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.519137 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 03:42:12.550114 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.519138 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:42:12.550114 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.519154 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 03:42:12.550114 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.519170 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 03:42:12.550114 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.519184 2566 projected.go:196] Error preparing data for projected volume kube-api-access-4dd75 for pod openshift-network-diagnostics/network-check-target-vkw66: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:42:12.550114 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.519198 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret podName:69b8b6fa-1fda-4c62-81d1-c9b27231a274 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:13.519180908 +0000 UTC m=+4.150840979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret") pod "global-pull-secret-syncer-k9f2k" (UID: "69b8b6fa-1fda-4c62-81d1-c9b27231a274") : object "kube-system"/"original-pull-secret" not registered Jul 09 03:42:12.550114 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.519219 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs podName:c275241c-186f-492a-952a-cb6ed44c560f nodeName:}" failed. No retries permitted until 2026-07-09 03:42:13.519209508 +0000 UTC m=+4.150869575 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs") pod "network-metrics-daemon-r9c5k" (UID: "c275241c-186f-492a-952a-cb6ed44c560f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:42:12.550114 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:12.519233 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75 podName:ecce4016-016f-4060-8fcc-d350522156ad nodeName:}" failed. No retries permitted until 2026-07-09 03:42:13.519225946 +0000 UTC m=+4.150886013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dd75" (UniqueName: "kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75") pod "network-check-target-vkw66" (UID: "ecce4016-016f-4060-8fcc-d350522156ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:42:12.863947 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.863571 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 03:37:12 +0000 UTC" deadline="2028-04-11 15:02:29.872348146 +0000 UTC" Jul 09 03:42:12.863947 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.863736 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15419h20m17.008618839s" Jul 09 03:42:12.869975 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.869933 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h5d9p" event={"ID":"758f53c8-f537-4587-9523-3c9ccedafd4a","Type":"ContainerStarted","Data":"f9a865d3a6bd80a113c143153e680216e8cf47a56cdf0a909a962ba35403f458"} Jul 09 03:42:12.873624 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.873469 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nfr48" event={"ID":"175280b2-1ecd-4cc1-a96a-5a9a13207aef","Type":"ContainerStarted","Data":"d9104dfc27f242459781c4e924adb13c21883703b5c02f413c20d96c6a074942"} Jul 09 03:42:12.875107 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.875011 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mf9s5" event={"ID":"6ac5e518-8402-4297-8ce7-4f083d9c2df2","Type":"ContainerStarted","Data":"95ebb0e3dc16e7e34f6b4cde89ab9cd6713ee9770bc92dbf9ec039275f4a6699"} Jul 09 03:42:12.876994 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.876960 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" event={"ID":"e894676b-96c1-48de-b161-d9a92e86fcf6","Type":"ContainerStarted","Data":"b766d3c5b40e961180981809ce6e9ea67406d702e5694a251f2858538b7c07b6"} Jul 09 03:42:12.878501 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.878468 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" event={"ID":"03b1ba2d-7000-46e5-af08-6977005ca09d","Type":"ContainerStarted","Data":"3ac146a19ceaf6e8444c3ef57f5fb80595367dfe8c072c7158cc240c9864c85e"} Jul 09 03:42:12.880219 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.880147 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6dggs" event={"ID":"6da3145c-5f91-44ca-8465-ce298a4ee12e","Type":"ContainerStarted","Data":"27503fa3fa4d8a97b3f4c79d93226b8176e67cdb179247471899ab10e48fc108"} Jul 09 03:42:12.882123 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.882037 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" event={"ID":"c205b758-9f45-4a64-be35-efa418109da9","Type":"ContainerStarted","Data":"84ec6e9885832c16dc49ad7316d3f69bdf97bda063e8996b9a22a498baa9c6b1"} Jul 09 03:42:12.883554 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.883485 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nk6tf" event={"ID":"167f4240-06e8-45a9-8770-56c395498f7d","Type":"ContainerStarted","Data":"0f274a147b1756da2335f72bc40f448cb7d935f6d7168080f05b6aeeac4be5d7"} Jul 09 03:42:12.885884 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:12.885850 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkpv2" event={"ID":"e0abbca0-fa5c-40f8-be6d-225daabb1d4e","Type":"ContainerStarted","Data":"e389b716b77267daf91c91f63b37bb1afca09ff5b0ede2e40809bcd2cc60d42c"} Jul 09 03:42:13.528414 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:13.528375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:13.529095 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:13.528448 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dd75\" (UniqueName: \"kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75\") pod \"network-check-target-vkw66\" (UID: \"ecce4016-016f-4060-8fcc-d350522156ad\") " pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:13.529095 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:13.528487 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs\") pod \"network-metrics-daemon-r9c5k\" (UID: \"c275241c-186f-492a-952a-cb6ed44c560f\") " pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:13.529095 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:13.528560 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 03:42:13.529095 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:13.528601 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:42:13.529095 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:13.528642 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret podName:69b8b6fa-1fda-4c62-81d1-c9b27231a274 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:15.528619239 +0000 UTC m=+6.160279311 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret") pod "global-pull-secret-syncer-k9f2k" (UID: "69b8b6fa-1fda-4c62-81d1-c9b27231a274") : object "kube-system"/"original-pull-secret" not registered Jul 09 03:42:13.529095 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:13.528678 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs podName:c275241c-186f-492a-952a-cb6ed44c560f nodeName:}" failed. No retries permitted until 2026-07-09 03:42:15.528652577 +0000 UTC m=+6.160312645 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs") pod "network-metrics-daemon-r9c5k" (UID: "c275241c-186f-492a-952a-cb6ed44c560f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:42:13.529095 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:13.528742 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 03:42:13.529095 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:13.528757 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 03:42:13.529095 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:13.528770 2566 projected.go:196] Error preparing data for projected volume kube-api-access-4dd75 for pod openshift-network-diagnostics/network-check-target-vkw66: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:42:13.529095 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:13.528826 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75 podName:ecce4016-016f-4060-8fcc-d350522156ad nodeName:}" failed. No retries permitted until 2026-07-09 03:42:15.528797867 +0000 UTC m=+6.160457951 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dd75" (UniqueName: "kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75") pod "network-check-target-vkw66" (UID: "ecce4016-016f-4060-8fcc-d350522156ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:42:13.862324 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:13.862236 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:13.862481 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:13.862395 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:13.862908 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:13.862881 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:13.863037 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:13.862991 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:13.863107 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:13.863086 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:13.863254 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:13.863181 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:13.864151 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:13.864115 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 03:37:12 +0000 UTC" deadline="2028-04-20 18:03:08.36758023 +0000 UTC" Jul 09 03:42:13.864260 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:13.864152 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15638h20m54.50343324s" Jul 09 03:42:15.549962 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:15.549333 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:15.549962 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:15.549417 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dd75\" (UniqueName: \"kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75\") pod \"network-check-target-vkw66\" (UID: \"ecce4016-016f-4060-8fcc-d350522156ad\") " pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:15.549962 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:15.549461 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs\") pod \"network-metrics-daemon-r9c5k\" (UID: \"c275241c-186f-492a-952a-cb6ed44c560f\") " pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:15.549962 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:15.549594 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:42:15.549962 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:15.549661 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs podName:c275241c-186f-492a-952a-cb6ed44c560f nodeName:}" failed. No retries permitted until 2026-07-09 03:42:19.549641082 +0000 UTC m=+10.181301169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs") pod "network-metrics-daemon-r9c5k" (UID: "c275241c-186f-492a-952a-cb6ed44c560f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:42:15.549962 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:15.549747 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 03:42:15.549962 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:15.549783 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret podName:69b8b6fa-1fda-4c62-81d1-c9b27231a274 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:19.549770554 +0000 UTC m=+10.181430625 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret") pod "global-pull-secret-syncer-k9f2k" (UID: "69b8b6fa-1fda-4c62-81d1-c9b27231a274") : object "kube-system"/"original-pull-secret" not registered Jul 09 03:42:15.549962 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:15.549867 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 03:42:15.549962 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:15.549886 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 03:42:15.549962 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:15.549900 2566 projected.go:196] Error preparing data for projected volume kube-api-access-4dd75 for pod openshift-network-diagnostics/network-check-target-vkw66: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:42:15.549962 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:15.549935 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75 podName:ecce4016-016f-4060-8fcc-d350522156ad nodeName:}" failed. No retries permitted until 2026-07-09 03:42:19.549924522 +0000 UTC m=+10.181584603 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dd75" (UniqueName: "kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75") pod "network-check-target-vkw66" (UID: "ecce4016-016f-4060-8fcc-d350522156ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:42:15.861484 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:15.861380 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:15.861656 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:15.861555 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:15.861656 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:15.861651 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:15.861807 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:15.861753 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:15.861872 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:15.861816 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:15.861924 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:15.861896 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:17.861464 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:17.860701 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:17.861464 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:17.860854 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:17.861464 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:17.861274 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:17.861464 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:17.861379 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:17.862148 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:17.861854 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:17.862148 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:17.861967 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:19.588933 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:19.588853 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dd75\" (UniqueName: \"kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75\") pod \"network-check-target-vkw66\" (UID: \"ecce4016-016f-4060-8fcc-d350522156ad\") " pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:19.589417 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:19.589205 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs\") pod \"network-metrics-daemon-r9c5k\" (UID: \"c275241c-186f-492a-952a-cb6ed44c560f\") " pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:19.589417 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:19.589295 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:19.589534 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:19.589451 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 03:42:19.589534 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:19.589498 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:42:19.589534 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:19.589525 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret podName:69b8b6fa-1fda-4c62-81d1-c9b27231a274 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:27.5895093 +0000 UTC m=+18.221169366 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret") pod "global-pull-secret-syncer-k9f2k" (UID: "69b8b6fa-1fda-4c62-81d1-c9b27231a274") : object "kube-system"/"original-pull-secret" not registered Jul 09 03:42:19.589681 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:19.589584 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs podName:c275241c-186f-492a-952a-cb6ed44c560f nodeName:}" failed. No retries permitted until 2026-07-09 03:42:27.589567135 +0000 UTC m=+18.221227206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs") pod "network-metrics-daemon-r9c5k" (UID: "c275241c-186f-492a-952a-cb6ed44c560f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:42:19.589822 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:19.589805 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 03:42:19.589897 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:19.589825 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 03:42:19.589897 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:19.589843 2566 projected.go:196] Error preparing data for projected volume kube-api-access-4dd75 for pod openshift-network-diagnostics/network-check-target-vkw66: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:42:19.589897 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:19.589878 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75 podName:ecce4016-016f-4060-8fcc-d350522156ad nodeName:}" failed. No retries permitted until 2026-07-09 03:42:27.589867691 +0000 UTC m=+18.221527757 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dd75" (UniqueName: "kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75") pod "network-check-target-vkw66" (UID: "ecce4016-016f-4060-8fcc-d350522156ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:42:19.860293 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:19.860185 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:19.860293 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:19.860218 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:19.860537 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:19.860333 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:19.860913 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:19.860782 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:19.860913 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:19.860826 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:19.860913 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:19.860887 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:21.858642 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:21.858611 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:21.859104 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:21.858611 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:21.859104 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:21.858615 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:21.859104 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:21.858783 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:21.859104 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:21.858875 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:21.859104 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:21.858997 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:23.859074 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:23.858851 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:23.859074 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:23.858856 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:23.859074 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:23.859006 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:23.859074 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:23.858999 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:23.859843 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:23.859101 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:23.859843 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:23.859177 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:25.859109 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:25.859073 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:25.859528 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:25.859073 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:25.859528 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:25.859205 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:25.859528 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:25.859073 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:25.859528 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:25.859254 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:25.859528 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:25.859391 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:27.652043 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:27.651948 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:27.652043 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:27.652011 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dd75\" (UniqueName: \"kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75\") pod \"network-check-target-vkw66\" (UID: \"ecce4016-016f-4060-8fcc-d350522156ad\") " pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:27.652617 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:27.652051 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs\") pod \"network-metrics-daemon-r9c5k\" (UID: \"c275241c-186f-492a-952a-cb6ed44c560f\") " pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:27.652617 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:27.652116 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 03:42:27.652617 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:27.652151 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:42:27.652617 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:27.652198 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret podName:69b8b6fa-1fda-4c62-81d1-c9b27231a274 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:43.652179451 +0000 UTC m=+34.283839515 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret") pod "global-pull-secret-syncer-k9f2k" (UID: "69b8b6fa-1fda-4c62-81d1-c9b27231a274") : object "kube-system"/"original-pull-secret" not registered Jul 09 03:42:27.652617 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:27.652198 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 03:42:27.652617 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:27.652214 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs podName:c275241c-186f-492a-952a-cb6ed44c560f nodeName:}" failed. No retries permitted until 2026-07-09 03:42:43.652206269 +0000 UTC m=+34.283866332 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs") pod "network-metrics-daemon-r9c5k" (UID: "c275241c-186f-492a-952a-cb6ed44c560f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:42:27.652617 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:27.652225 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 03:42:27.652617 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:27.652239 2566 projected.go:196] Error preparing data for projected volume kube-api-access-4dd75 for pod openshift-network-diagnostics/network-check-target-vkw66: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:42:27.652617 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:27.652294 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75 podName:ecce4016-016f-4060-8fcc-d350522156ad nodeName:}" failed. No retries permitted until 2026-07-09 03:42:43.652275443 +0000 UTC m=+34.283935549 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dd75" (UniqueName: "kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75") pod "network-check-target-vkw66" (UID: "ecce4016-016f-4060-8fcc-d350522156ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:42:27.858787 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:27.858755 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:27.858967 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:27.858755 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:27.858967 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:27.858883 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:27.859101 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:27.858764 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:27.859101 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:27.858977 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:27.859200 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:27.859161 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:29.859384 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:29.859337 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:29.859875 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:29.859447 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:29.859875 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:29.859547 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:29.859875 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:29.859676 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:29.859875 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:29.859712 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:29.859875 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:29.859824 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:31.858666 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:31.858612 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:31.859046 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:31.858612 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:31.859046 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:31.858623 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:31.859046 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:31.858782 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:31.859046 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:31.858856 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:31.859046 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:31.858950 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:32.929533 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.929376 2566 generic.go:356] "Generic (PLEG): container finished" podID="2f94e666b22f673976f4d41923aab2d9" containerID="33e3357576135a128fe7233cba647bb8e11d35c894b58b082d2e25e12e8ecedb" exitCode=0 Jul 09 03:42:32.930202 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.929449 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal" event={"ID":"2f94e666b22f673976f4d41923aab2d9","Type":"ContainerDied","Data":"33e3357576135a128fe7233cba647bb8e11d35c894b58b082d2e25e12e8ecedb"} Jul 09 03:42:32.930951 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.930928 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nfr48" event={"ID":"175280b2-1ecd-4cc1-a96a-5a9a13207aef","Type":"ContainerStarted","Data":"10396148a91b12361a29d425797f3b1a40b7f9c877cf8cf5656dc37664b59b52"} Jul 09 03:42:32.932290 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.932269 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mf9s5" event={"ID":"6ac5e518-8402-4297-8ce7-4f083d9c2df2","Type":"ContainerStarted","Data":"11ff5979d267e3388b03dc533aad6f29521ce6dd26e3f338fc869335f9abbb59"} Jul 09 03:42:32.933523 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.933498 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" event={"ID":"e894676b-96c1-48de-b161-d9a92e86fcf6","Type":"ContainerStarted","Data":"cbc8e42cda07df3b9b259643bdd683cb8055703ec3e437c73bfe1220d5a61377"} Jul 09 03:42:32.934784 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.934763 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-230.ec2.internal" event={"ID":"e59dcb1db041284efb34c27cebcb17d3","Type":"ContainerStarted","Data":"e6a7f8d52fce98a8bf296af88ec4164912e1f491fa8eaa65c999b7954903e5ac"} Jul 09 03:42:32.937418 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.937400 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" event={"ID":"03b1ba2d-7000-46e5-af08-6977005ca09d","Type":"ContainerStarted","Data":"4b3fb1af05073f27aa54f92083880bc52c903b3fdfa69829a3947f022d8642fc"} Jul 09 03:42:32.937518 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.937464 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" event={"ID":"03b1ba2d-7000-46e5-af08-6977005ca09d","Type":"ContainerStarted","Data":"92df74bd54a29e143f07d8eb3c6e032af7dd47a98f6345adaa802124a36dc501"} Jul 09 03:42:32.937518 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.937495 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" event={"ID":"03b1ba2d-7000-46e5-af08-6977005ca09d","Type":"ContainerStarted","Data":"12f05b87dcaa33e1b9e1259fd517c252b81f878475ae71b05c2ed63bfce206e2"} Jul 09 03:42:32.937609 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.937535 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" event={"ID":"03b1ba2d-7000-46e5-af08-6977005ca09d","Type":"ContainerStarted","Data":"0f56b901282a384586a89d49fdd6738cf3b3cb6e4e46c64cb1e1314f8d8595f9"} Jul 09 03:42:32.937609 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.937574 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" event={"ID":"03b1ba2d-7000-46e5-af08-6977005ca09d","Type":"ContainerStarted","Data":"c82c41dc0858e781da2b1102a99744b3b69ae315e50ac417f422223f2e9a54ca"} Jul 09 03:42:32.937709 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.937621 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" event={"ID":"03b1ba2d-7000-46e5-af08-6977005ca09d","Type":"ContainerStarted","Data":"fd3dd8c993d88cc95812be8560b6a0941aa1e7a0e58a1a381ad88ed4d2d5b35b"} Jul 09 03:42:32.938678 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.938637 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6dggs" event={"ID":"6da3145c-5f91-44ca-8465-ce298a4ee12e","Type":"ContainerStarted","Data":"8bcfc6ea3e60f4dc38f4d97ed7ba20b08672c4624df81644778340f02b08a7ea"} Jul 09 03:42:32.940050 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.940025 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" event={"ID":"c205b758-9f45-4a64-be35-efa418109da9","Type":"ContainerStarted","Data":"4f360fdc5578e3ea353674106153f2f59e7467cd832e92b98804a55e2aaad0b1"} Jul 09 03:42:32.941295 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.941277 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nk6tf" event={"ID":"167f4240-06e8-45a9-8770-56c395498f7d","Type":"ContainerStarted","Data":"3dfe3000e217f821761d261d10f637d3fa1abfcd708c49cf6f8918ee66f0d6d2"} Jul 09 03:42:32.942594 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.942571 2566 generic.go:356] "Generic (PLEG): container finished" podID="e0abbca0-fa5c-40f8-be6d-225daabb1d4e" containerID="52b2f2448e7f90ffc54347b6a191faf7432d9ef52a4787a442e787a07dfe2fd1" exitCode=0 Jul 09 03:42:32.942686 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.942606 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkpv2" event={"ID":"e0abbca0-fa5c-40f8-be6d-225daabb1d4e","Type":"ContainerDied","Data":"52b2f2448e7f90ffc54347b6a191faf7432d9ef52a4787a442e787a07dfe2fd1"} Jul 09 03:42:32.947311 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.947271 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nfr48" podStartSLOduration=12.94847547 podStartE2EDuration="22.947259611s" totalImagesPullingTime="9.998784141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:32.94689467 +0000 UTC m=+23.578554756" watchObservedRunningTime="2026-07-09 03:42:32.947259611 +0000 UTC m=+23.578919697" Jul 09 03:42:32.984733 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.984685 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nk6tf" podStartSLOduration=3.206055437 podStartE2EDuration="22.984669943s" totalImagesPullingTime="19.778614506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:32.984427163 +0000 UTC m=+23.616087248" watchObservedRunningTime="2026-07-09 03:42:32.984669943 +0000 UTC m=+23.616330026" Jul 09 03:42:32.999605 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:32.999558 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gqxlk" podStartSLOduration=3.281377537 podStartE2EDuration="22.999541906s" totalImagesPullingTime="19.718164369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:32.999208086 +0000 UTC m=+23.630868172" watchObservedRunningTime="2026-07-09 03:42:32.999541906 +0000 UTC m=+23.631201992" Jul 09 03:42:33.013104 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:33.013060 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mf9s5" podStartSLOduration=12.997892586 podStartE2EDuration="23.013046928s" totalImagesPullingTime="10.015154342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:33.012421424 +0000 UTC m=+23.644081512" watchObservedRunningTime="2026-07-09 03:42:33.013046928 +0000 UTC m=+23.644707013" Jul 09 03:42:33.049436 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:33.049383 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6dggs" podStartSLOduration=3.332742807 podStartE2EDuration="23.0493675s" totalImagesPullingTime="19.716624693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:33.033714544 +0000 UTC m=+23.665374629" watchObservedRunningTime="2026-07-09 03:42:33.0493675 +0000 UTC m=+23.681027588" Jul 09 03:42:33.859417 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:33.859333 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:33.859417 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:33.859368 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:33.859857 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:33.859337 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:33.859857 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:33.859480 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:33.859857 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:33.859544 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:33.859857 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:33.859599 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:33.946980 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:33.946911 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h5d9p" event={"ID":"758f53c8-f537-4587-9523-3c9ccedafd4a","Type":"ContainerStarted","Data":"87f38ad80d4c37ed9db0bfd6c6259b0a66276ff49f29df9c6e5b4e7c36fc4719"} Jul 09 03:42:33.961450 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:33.961405 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h5d9p" podStartSLOduration=3.256114466 podStartE2EDuration="22.961389789s" totalImagesPullingTime="19.705275323s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:33.960984923 +0000 UTC m=+24.592645009" watchObservedRunningTime="2026-07-09 03:42:33.961389789 +0000 UTC m=+24.593049875" Jul 09 03:42:33.961592 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:33.961480 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-230.ec2.internal" podStartSLOduration=21.961474041 podStartE2EDuration="21.961474041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:42:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:33.048949459 +0000 UTC m=+23.680609547" watchObservedRunningTime="2026-07-09 03:42:33.961474041 +0000 UTC m=+24.593134126" Jul 09 03:42:34.295146 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:34.295105 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 03:42:34.879278 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:34.879195 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T03:42:34.295138641Z","UUID":"a9ddcded-b7da-4784-9283-5d48b59fba37","Handler":null,"Name":"","Endpoint":""} Jul 09 03:42:34.880622 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:34.880602 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 03:42:34.880738 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:34.880661 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 03:42:34.953438 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:34.953400 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal" event={"ID":"2f94e666b22f673976f4d41923aab2d9","Type":"ContainerStarted","Data":"82e432f91a55c025389d5663f74ac5c778ac664a85c19c4e4fca724cadcd6c2f"} Jul 09 03:42:34.957121 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:34.957090 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" event={"ID":"e894676b-96c1-48de-b161-d9a92e86fcf6","Type":"ContainerStarted","Data":"b88a45faa8187495b59174b3f6eae8bc801e7d82fe144b3970660644efc833f0"} Jul 09 03:42:34.961435 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:34.961402 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" event={"ID":"03b1ba2d-7000-46e5-af08-6977005ca09d","Type":"ContainerStarted","Data":"bf027517261ba395671d6942c90787186efb21d1b8f79cc8fcb56a44ec96d1eb"} Jul 09 03:42:34.968618 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:34.968457 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-230.ec2.internal" podStartSLOduration=22.96843688 podStartE2EDuration="22.96843688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:42:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:34.967439654 +0000 UTC m=+25.599099753" watchObservedRunningTime="2026-07-09 03:42:34.96843688 +0000 UTC m=+25.600096970" Jul 09 03:42:35.859249 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:35.859209 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:35.859440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:35.859208 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:35.859440 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:35.859339 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:35.859440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:35.859209 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:35.859440 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:35.859393 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:35.859681 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:35.859473 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:35.965396 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:35.965315 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" event={"ID":"e894676b-96c1-48de-b161-d9a92e86fcf6","Type":"ContainerStarted","Data":"1e908299ddebf46ac66517d29f98468836eeca935edfd2425e59da206e719e6a"} Jul 09 03:42:35.982287 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:35.982230 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8z96v" podStartSLOduration=3.602479592 podStartE2EDuration="25.982215349s" totalImagesPullingTime="22.379735757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:42:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:35.981999929 +0000 UTC m=+26.613660014" watchObservedRunningTime="2026-07-09 03:42:35.982215349 +0000 UTC m=+26.613875432" Jul 09 03:42:36.947009 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:36.946962 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nfr48" Jul 09 03:42:36.947760 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:36.947737 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nfr48" Jul 09 03:42:36.968682 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:36.968577 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nfr48" Jul 09 03:42:36.969246 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:36.968946 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nfr48" Jul 09 03:42:37.859148 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:37.859114 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:37.859148 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:37.859133 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:37.859413 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:37.859253 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:37.859413 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:37.859281 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:37.859413 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:37.859373 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:37.859579 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:37.859474 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:38.976626 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:38.976355 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" event={"ID":"03b1ba2d-7000-46e5-af08-6977005ca09d","Type":"ContainerStarted","Data":"643a938401b94d9ce32bf148fce36f11c8be6e523bd127b6580c1ff70951819d"} Jul 09 03:42:38.977378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:38.976642 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:38.977378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:38.976698 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:38.978223 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:38.978198 2566 generic.go:356] "Generic (PLEG): container finished" podID="e0abbca0-fa5c-40f8-be6d-225daabb1d4e" containerID="c16986c16263d1a4430a621840e55896447c45b574b20652b7e3c4487042ad3b" exitCode=0 Jul 09 03:42:38.978348 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:38.978285 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkpv2" event={"ID":"e0abbca0-fa5c-40f8-be6d-225daabb1d4e","Type":"ContainerDied","Data":"c16986c16263d1a4430a621840e55896447c45b574b20652b7e3c4487042ad3b"} Jul 09 03:42:38.993164 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:38.993136 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:39.001447 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:39.001403 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" podStartSLOduration=8.061072328 podStartE2EDuration="28.001389856s" totalImagesPullingTime="19.940317528s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:38.999910558 +0000 UTC m=+29.631570645" watchObservedRunningTime="2026-07-09 03:42:39.001389856 +0000 UTC m=+29.633049941" Jul 09 03:42:39.860066 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:39.860028 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:39.860318 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:39.860129 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:39.860318 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:39.860142 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:39.860318 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:39.860254 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:39.860318 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:39.860318 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:39.860508 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:39.860390 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:39.981183 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:39.981150 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:39.996111 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:39.996084 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:42:40.719916 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:40.719813 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r9c5k"] Jul 09 03:42:40.720079 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:40.719961 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:40.720148 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:40.720107 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:40.722760 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:40.722728 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k9f2k"] Jul 09 03:42:40.722904 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:40.722830 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:40.722959 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:40.722937 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:40.723588 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:40.723548 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vkw66"] Jul 09 03:42:40.723730 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:40.723635 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:40.723784 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:40.723728 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:40.985000 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:40.984921 2566 generic.go:356] "Generic (PLEG): container finished" podID="e0abbca0-fa5c-40f8-be6d-225daabb1d4e" containerID="b71abc1441cc62dc0c7df8dcd1f78afcaf7630902487581d006afc02dc4a8bf9" exitCode=0 Jul 09 03:42:40.985343 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:40.985007 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkpv2" event={"ID":"e0abbca0-fa5c-40f8-be6d-225daabb1d4e","Type":"ContainerDied","Data":"b71abc1441cc62dc0c7df8dcd1f78afcaf7630902487581d006afc02dc4a8bf9"} Jul 09 03:42:41.990095 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:41.990057 2566 generic.go:356] "Generic (PLEG): container finished" podID="e0abbca0-fa5c-40f8-be6d-225daabb1d4e" containerID="c0dbf903b2d3d019ccece106b5faa0fa24daed5addcc5b86d57c86eb9f48b010" exitCode=0 Jul 09 03:42:41.990628 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:41.990111 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkpv2" event={"ID":"e0abbca0-fa5c-40f8-be6d-225daabb1d4e","Type":"ContainerDied","Data":"c0dbf903b2d3d019ccece106b5faa0fa24daed5addcc5b86d57c86eb9f48b010"} Jul 09 03:42:42.858456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:42.858427 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:42.858665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:42.858428 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:42.858665 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:42.858540 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:42.858665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:42.858574 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:42.858840 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:42.858695 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:42.858840 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:42.858777 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:43.667800 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:43.667750 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dd75\" (UniqueName: \"kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75\") pod \"network-check-target-vkw66\" (UID: \"ecce4016-016f-4060-8fcc-d350522156ad\") " pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:43.667800 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:43.667810 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs\") pod \"network-metrics-daemon-r9c5k\" (UID: \"c275241c-186f-492a-952a-cb6ed44c560f\") " pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:43.668269 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:43.667841 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:43.668269 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:43.667923 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 03:42:43.668269 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:43.667932 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 03:42:43.668269 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:43.667959 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 03:42:43.668269 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:43.667970 2566 projected.go:196] Error preparing data for projected volume kube-api-access-4dd75 for pod openshift-network-diagnostics/network-check-target-vkw66: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:42:43.668269 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:43.667930 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:42:43.668269 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:43.667970 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret podName:69b8b6fa-1fda-4c62-81d1-c9b27231a274 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:15.667957559 +0000 UTC m=+66.299617623 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret") pod "global-pull-secret-syncer-k9f2k" (UID: "69b8b6fa-1fda-4c62-81d1-c9b27231a274") : object "kube-system"/"original-pull-secret" not registered Jul 09 03:42:43.668269 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:43.668045 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75 podName:ecce4016-016f-4060-8fcc-d350522156ad nodeName:}" failed. No retries permitted until 2026-07-09 03:43:15.668021763 +0000 UTC m=+66.299681830 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4dd75" (UniqueName: "kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75") pod "network-check-target-vkw66" (UID: "ecce4016-016f-4060-8fcc-d350522156ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:42:43.668269 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:43.668070 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs podName:c275241c-186f-492a-952a-cb6ed44c560f nodeName:}" failed. No retries permitted until 2026-07-09 03:43:15.66805914 +0000 UTC m=+66.299719208 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs") pod "network-metrics-daemon-r9c5k" (UID: "c275241c-186f-492a-952a-cb6ed44c560f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:42:44.859091 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:44.859057 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:44.859580 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:44.859100 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:44.859580 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:44.859073 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:44.859580 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:44.859179 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vkw66" podUID="ecce4016-016f-4060-8fcc-d350522156ad" Jul 09 03:42:44.859580 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:44.859231 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r9c5k" podUID="c275241c-186f-492a-952a-cb6ed44c560f" Jul 09 03:42:44.859580 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:44.859301 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9f2k" podUID="69b8b6fa-1fda-4c62-81d1-c9b27231a274" Jul 09 03:42:45.647746 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.647716 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-230.ec2.internal" event="NodeReady" Jul 09 03:42:45.647909 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.647842 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 03:42:45.678688 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.678627 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ddhcf"] Jul 09 03:42:45.681395 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.681352 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.684335 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.684309 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f8c949fc8-jw5ks"] Jul 09 03:42:45.685119 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.684588 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 09 03:42:45.685119 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.684607 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 09 03:42:45.685119 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.684628 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 03:42:45.685119 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.684638 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 03:42:45.685119 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.684984 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-2vptz\"" Jul 09 03:42:45.687140 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.687111 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7dfb6bc4d7-d7vz4"] Jul 09 03:42:45.687303 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.687261 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.689951 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.689930 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2tpsp"] Jul 09 03:42:45.690076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.690057 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:45.693943 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.691095 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 03:42:45.693943 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.691612 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 03:42:45.693943 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.691708 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 03:42:45.693943 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.692989 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nqczr\"" Jul 09 03:42:45.695569 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.695261 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 03:42:45.698144 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.698125 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 03:42:45.698581 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.698239 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 09 03:42:45.698775 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.698757 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 09 03:42:45.700448 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.699976 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 09 03:42:45.702282 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.702260 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 09 03:42:45.702380 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.702354 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4zwkd\"" Jul 09 03:42:45.702785 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.702679 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 09 03:42:45.706203 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.706179 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq"] Jul 09 03:42:45.706306 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.706279 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 03:42:45.706488 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.706465 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2tpsp" Jul 09 03:42:45.711114 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.709472 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-44xps\"" Jul 09 03:42:45.711114 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.709570 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 09 03:42:45.711114 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.710050 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-gskrv"] Jul 09 03:42:45.711114 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.710279 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:42:45.712779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.712530 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 09 03:42:45.713720 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.713694 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-qx2k9\"" Jul 09 03:42:45.713937 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.713922 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 03:42:45.714032 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.714013 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 03:42:45.714261 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.714243 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 09 03:42:45.714261 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.714257 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn"] Jul 09 03:42:45.714588 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.714567 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:42:45.715166 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.715115 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 09 03:42:45.717365 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.717347 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 09 03:42:45.717628 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.717606 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 09 03:42:45.717628 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.717615 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 09 03:42:45.717839 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.717745 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 09 03:42:45.717959 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.717935 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8"] Jul 09 03:42:45.718066 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.717989 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-hdlnh\"" Jul 09 03:42:45.718066 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.718026 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" Jul 09 03:42:45.720539 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.720518 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 09 03:42:45.721027 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.721006 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 09 03:42:45.721690 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.721671 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-bm4nv\"" Jul 09 03:42:45.721782 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.721737 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ddhcf"] Jul 09 03:42:45.721782 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.721778 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2tpsp"] Jul 09 03:42:45.721886 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.721800 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8"] Jul 09 03:42:45.721886 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.721817 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-gskrv"] Jul 09 03:42:45.721886 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.721835 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7dfb6bc4d7-d7vz4"] Jul 09 03:42:45.721886 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.721875 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h"] Jul 09 03:42:45.722195 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.722179 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" Jul 09 03:42:45.722366 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.722349 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 09 03:42:45.724574 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.724554 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 09 03:42:45.725599 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.725471 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 09 03:42:45.725773 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.725716 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 09 03:42:45.725913 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.725882 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f8c949fc8-jw5ks"] Jul 09 03:42:45.726115 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.726096 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" Jul 09 03:42:45.726203 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.725976 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 09 03:42:45.726366 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.726049 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-wt9ms\"" Jul 09 03:42:45.726520 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.726099 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-vh66t"] Jul 09 03:42:45.726630 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.725915 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 09 03:42:45.731327 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.731305 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 09 03:42:45.733921 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.731303 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 09 03:42:45.734187 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.731309 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 09 03:42:45.734346 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.731372 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 09 03:42:45.734449 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.731865 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-q42jw\"" Jul 09 03:42:45.736821 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.736797 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq"] Jul 09 03:42:45.736907 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.736836 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h"] Jul 09 03:42:45.736907 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.736864 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b64b4"] Jul 09 03:42:45.737108 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.737075 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vh66t" Jul 09 03:42:45.740235 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.740212 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 03:42:45.741221 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.741185 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 03:42:45.741397 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.741268 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-59bbh\"" Jul 09 03:42:45.741540 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.741524 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mzcvn"] Jul 09 03:42:45.741736 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.741715 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b64b4" Jul 09 03:42:45.744107 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.744086 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 03:42:45.744807 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.744695 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 03:42:45.744807 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.744709 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-k4rlp\"" Jul 09 03:42:45.744807 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.744788 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 03:42:45.745416 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.745397 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-vh66t"] Jul 09 03:42:45.745527 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.745433 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn"] Jul 09 03:42:45.745527 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.745457 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b64b4"] Jul 09 03:42:45.745527 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.745475 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mzcvn"] Jul 09 03:42:45.745727 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.745569 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:45.748033 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.748011 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 03:42:45.748179 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.748167 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 03:42:45.748268 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.748245 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hwlx6\"" Jul 09 03:42:45.781706 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.781677 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ljdr\" (UniqueName: \"kubernetes.io/projected/d7993763-77e3-47b3-a32d-aa4b7c406413-kube-api-access-4ljdr\") pod \"console-operator-68b757fb67-gskrv\" (UID: \"d7993763-77e3-47b3-a32d-aa4b7c406413\") " pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:42:45.781876 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.781713 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88d0565d-8fda-4829-833e-43b2b5e36610-trusted-ca\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.781876 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.781744 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x79j\" (UniqueName: \"kubernetes.io/projected/f8a79e10-841e-4cbc-83f5-565ac6b5dfcc-kube-api-access-6x79j\") pod \"volume-data-source-validator-5d456fdbcd-2tpsp\" (UID: \"f8a79e10-841e-4cbc-83f5-565ac6b5dfcc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2tpsp" Jul 09 03:42:45.781876 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.781810 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2c564d7-7702-4488-8faa-a7c27f1608c0-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-tmxz8\" (UID: \"f2c564d7-7702-4488-8faa-a7c27f1608c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" Jul 09 03:42:45.782043 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.781878 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a92ef34e-50ec-4d34-8d0f-a6e307d7a949-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-lbz6h\" (UID: \"a92ef34e-50ec-4d34-8d0f-a6e307d7a949\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" Jul 09 03:42:45.782043 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.781908 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88d0565d-8fda-4829-833e-43b2b5e36610-ca-trust-extracted\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.782043 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.781932 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71b0a743-fa60-46e6-9fde-965caa5bdf86-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.782043 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.781962 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:45.782043 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.781990 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7993763-77e3-47b3-a32d-aa4b7c406413-config\") pod \"console-operator-68b757fb67-gskrv\" (UID: \"d7993763-77e3-47b3-a32d-aa4b7c406413\") " pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:42:45.782043 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782021 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:45.782043 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782043 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcmxb\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-kube-api-access-qcmxb\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.782374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782066 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71b0a743-fa60-46e6-9fde-965caa5bdf86-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.782374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782097 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tldng\" (UniqueName: \"kubernetes.io/projected/27fcabd6-d124-4abd-8b2d-86cc903a5693-kube-api-access-tldng\") pod \"cluster-samples-operator-56fcf86bf-s29mn\" (UID: \"27fcabd6-d124-4abd-8b2d-86cc903a5693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" Jul 09 03:42:45.782374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782141 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-default-certificate\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:45.782374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782182 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7993763-77e3-47b3-a32d-aa4b7c406413-trusted-ca\") pod \"console-operator-68b757fb67-gskrv\" (UID: \"d7993763-77e3-47b3-a32d-aa4b7c406413\") " pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:42:45.782374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782203 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-bound-sa-token\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.782374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782226 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/71b0a743-fa60-46e6-9fde-965caa5bdf86-snapshots\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.782374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782252 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl4q5\" (UniqueName: \"kubernetes.io/projected/d01b29b9-9680-4c59-acad-460f17b275f4-kube-api-access-sl4q5\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:42:45.782374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782269 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a92ef34e-50ec-4d34-8d0f-a6e307d7a949-config\") pod \"service-ca-operator-5d46d9ffd4-lbz6h\" (UID: \"a92ef34e-50ec-4d34-8d0f-a6e307d7a949\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" Jul 09 03:42:45.782374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782286 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88d0565d-8fda-4829-833e-43b2b5e36610-image-registry-private-configuration\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.782374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782311 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88d0565d-8fda-4829-833e-43b2b5e36610-registry-certificates\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.782374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782328 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:42:45.782374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782353 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7993763-77e3-47b3-a32d-aa4b7c406413-serving-cert\") pod \"console-operator-68b757fb67-gskrv\" (UID: \"d7993763-77e3-47b3-a32d-aa4b7c406413\") " pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:42:45.782374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782370 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-stats-auth\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:45.782973 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782386 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88d0565d-8fda-4829-833e-43b2b5e36610-installation-pull-secrets\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.782973 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782401 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s29mn\" (UID: \"27fcabd6-d124-4abd-8b2d-86cc903a5693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" Jul 09 03:42:45.782973 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782429 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwws2\" (UniqueName: \"kubernetes.io/projected/93f8e14e-aea8-496e-b728-1554938d338a-kube-api-access-hwws2\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:45.782973 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782456 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stspc\" (UniqueName: \"kubernetes.io/projected/a92ef34e-50ec-4d34-8d0f-a6e307d7a949-kube-api-access-stspc\") pod \"service-ca-operator-5d46d9ffd4-lbz6h\" (UID: \"a92ef34e-50ec-4d34-8d0f-a6e307d7a949\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" Jul 09 03:42:45.782973 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782480 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d01b29b9-9680-4c59-acad-460f17b275f4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:42:45.782973 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782508 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f2c564d7-7702-4488-8faa-a7c27f1608c0-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-tmxz8\" (UID: \"f2c564d7-7702-4488-8faa-a7c27f1608c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" Jul 09 03:42:45.782973 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782535 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.782973 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782596 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gngd\" (UniqueName: \"kubernetes.io/projected/71b0a743-fa60-46e6-9fde-965caa5bdf86-kube-api-access-5gngd\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.782973 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782712 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71b0a743-fa60-46e6-9fde-965caa5bdf86-tmp\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.782973 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782746 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71b0a743-fa60-46e6-9fde-965caa5bdf86-serving-cert\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.782973 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.782786 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zgrv\" (UniqueName: \"kubernetes.io/projected/f2c564d7-7702-4488-8faa-a7c27f1608c0-kube-api-access-6zgrv\") pod \"kube-storage-version-migrator-operator-846bf668f7-tmxz8\" (UID: \"f2c564d7-7702-4488-8faa-a7c27f1608c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" Jul 09 03:42:45.883808 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.883721 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88d0565d-8fda-4829-833e-43b2b5e36610-ca-trust-extracted\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.883808 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.883759 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71b0a743-fa60-46e6-9fde-965caa5bdf86-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.883808 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.883803 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:45.884354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.883829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64ca5948-6243-4377-9c91-a3da567aff5e-config-volume\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:45.884354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.883862 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7993763-77e3-47b3-a32d-aa4b7c406413-config\") pod \"console-operator-68b757fb67-gskrv\" (UID: \"d7993763-77e3-47b3-a32d-aa4b7c406413\") " pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:42:45.884354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.883887 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnczj\" (UniqueName: \"kubernetes.io/projected/64ca5948-6243-4377-9c91-a3da567aff5e-kube-api-access-nnczj\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:45.884354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.883962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:45.884354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.883999 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcmxb\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-kube-api-access-qcmxb\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.884354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884038 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71b0a743-fa60-46e6-9fde-965caa5bdf86-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.884354 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:45.884094 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle podName:93f8e14e-aea8-496e-b728-1554938d338a nodeName:}" failed. No retries permitted until 2026-07-09 03:42:46.384068091 +0000 UTC m=+37.015728171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle") pod "router-default-7dfb6bc4d7-d7vz4" (UID: "93f8e14e-aea8-496e-b728-1554938d338a") : configmap references non-existent config key: service-ca.crt Jul 09 03:42:45.884354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884169 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88d0565d-8fda-4829-833e-43b2b5e36610-ca-trust-extracted\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.884354 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:45.884179 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 03:42:45.884354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884218 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64ca5948-6243-4377-9c91-a3da567aff5e-tmp-dir\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:45.884354 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:45.884237 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs podName:93f8e14e-aea8-496e-b728-1554938d338a nodeName:}" failed. No retries permitted until 2026-07-09 03:42:46.384223516 +0000 UTC m=+37.015883586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs") pod "router-default-7dfb6bc4d7-d7vz4" (UID: "93f8e14e-aea8-496e-b728-1554938d338a") : secret "router-metrics-certs-default" not found Jul 09 03:42:45.884354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884277 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tldng\" (UniqueName: \"kubernetes.io/projected/27fcabd6-d124-4abd-8b2d-86cc903a5693-kube-api-access-tldng\") pod \"cluster-samples-operator-56fcf86bf-s29mn\" (UID: \"27fcabd6-d124-4abd-8b2d-86cc903a5693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" Jul 09 03:42:45.884354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884312 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-default-certificate\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:45.884354 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884339 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884372 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7993763-77e3-47b3-a32d-aa4b7c406413-trusted-ca\") pod \"console-operator-68b757fb67-gskrv\" (UID: \"d7993763-77e3-47b3-a32d-aa4b7c406413\") " pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884396 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-bound-sa-token\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884422 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/71b0a743-fa60-46e6-9fde-965caa5bdf86-snapshots\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884445 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87-cert\") pod \"ingress-canary-b64b4\" (UID: \"5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87\") " pod="openshift-ingress-canary/ingress-canary-b64b4" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884467 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl4q5\" (UniqueName: \"kubernetes.io/projected/d01b29b9-9680-4c59-acad-460f17b275f4-kube-api-access-sl4q5\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884491 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a92ef34e-50ec-4d34-8d0f-a6e307d7a949-config\") pod \"service-ca-operator-5d46d9ffd4-lbz6h\" (UID: \"a92ef34e-50ec-4d34-8d0f-a6e307d7a949\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884517 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88d0565d-8fda-4829-833e-43b2b5e36610-image-registry-private-configuration\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884539 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88d0565d-8fda-4829-833e-43b2b5e36610-registry-certificates\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884560 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71b0a743-fa60-46e6-9fde-965caa5bdf86-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884567 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884611 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7993763-77e3-47b3-a32d-aa4b7c406413-serving-cert\") pod \"console-operator-68b757fb67-gskrv\" (UID: \"d7993763-77e3-47b3-a32d-aa4b7c406413\") " pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884636 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-stats-auth\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884687 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88d0565d-8fda-4829-833e-43b2b5e36610-installation-pull-secrets\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884707 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7993763-77e3-47b3-a32d-aa4b7c406413-config\") pod \"console-operator-68b757fb67-gskrv\" (UID: \"d7993763-77e3-47b3-a32d-aa4b7c406413\") " pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884713 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s29mn\" (UID: \"27fcabd6-d124-4abd-8b2d-86cc903a5693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" Jul 09 03:42:45.885076 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884762 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwws2\" (UniqueName: \"kubernetes.io/projected/93f8e14e-aea8-496e-b728-1554938d338a-kube-api-access-hwws2\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884787 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stspc\" (UniqueName: \"kubernetes.io/projected/a92ef34e-50ec-4d34-8d0f-a6e307d7a949-kube-api-access-stspc\") pod \"service-ca-operator-5d46d9ffd4-lbz6h\" (UID: \"a92ef34e-50ec-4d34-8d0f-a6e307d7a949\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:45.884794 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884812 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d01b29b9-9680-4c59-acad-460f17b275f4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884837 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f2c564d7-7702-4488-8faa-a7c27f1608c0-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-tmxz8\" (UID: \"f2c564d7-7702-4488-8faa-a7c27f1608c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884864 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:45.884875 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls podName:27fcabd6-d124-4abd-8b2d-86cc903a5693 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:46.384859243 +0000 UTC m=+37.016519308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-s29mn" (UID: "27fcabd6-d124-4abd-8b2d-86cc903a5693") : secret "samples-operator-tls" not found Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884909 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71b0a743-fa60-46e6-9fde-965caa5bdf86-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.884923 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gngd\" (UniqueName: \"kubernetes.io/projected/71b0a743-fa60-46e6-9fde-965caa5bdf86-kube-api-access-5gngd\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:45.884931 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:45.884945 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f8c949fc8-jw5ks: secret "image-registry-tls" not found Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:45.884982 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls podName:88d0565d-8fda-4829-833e-43b2b5e36610 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:46.384969557 +0000 UTC m=+37.016629634 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls") pod "image-registry-7f8c949fc8-jw5ks" (UID: "88d0565d-8fda-4829-833e-43b2b5e36610") : secret "image-registry-tls" not found Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.885066 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/71b0a743-fa60-46e6-9fde-965caa5bdf86-snapshots\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.885133 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7993763-77e3-47b3-a32d-aa4b7c406413-trusted-ca\") pod \"console-operator-68b757fb67-gskrv\" (UID: \"d7993763-77e3-47b3-a32d-aa4b7c406413\") " pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.885142 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a92ef34e-50ec-4d34-8d0f-a6e307d7a949-config\") pod \"service-ca-operator-5d46d9ffd4-lbz6h\" (UID: \"a92ef34e-50ec-4d34-8d0f-a6e307d7a949\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" Jul 09 03:42:45.885676 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:45.885219 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 03:42:45.887350 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.886958 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d01b29b9-9680-4c59-acad-460f17b275f4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:42:45.887350 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:45.887257 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls podName:d01b29b9-9680-4c59-acad-460f17b275f4 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:46.387234194 +0000 UTC m=+37.018894263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tl5sq" (UID: "d01b29b9-9680-4c59-acad-460f17b275f4") : secret "cluster-monitoring-operator-tls" not found Jul 09 03:42:45.887350 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.887346 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71b0a743-fa60-46e6-9fde-965caa5bdf86-tmp\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.887515 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.887389 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71b0a743-fa60-46e6-9fde-965caa5bdf86-serving-cert\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.887515 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.887423 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zgrv\" (UniqueName: \"kubernetes.io/projected/f2c564d7-7702-4488-8faa-a7c27f1608c0-kube-api-access-6zgrv\") pod \"kube-storage-version-migrator-operator-846bf668f7-tmxz8\" (UID: \"f2c564d7-7702-4488-8faa-a7c27f1608c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" Jul 09 03:42:45.887515 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.887507 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ljdr\" (UniqueName: \"kubernetes.io/projected/d7993763-77e3-47b3-a32d-aa4b7c406413-kube-api-access-4ljdr\") pod \"console-operator-68b757fb67-gskrv\" (UID: \"d7993763-77e3-47b3-a32d-aa4b7c406413\") " pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:42:45.887686 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.887544 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88d0565d-8fda-4829-833e-43b2b5e36610-trusted-ca\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.887686 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.887581 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x79j\" (UniqueName: \"kubernetes.io/projected/f8a79e10-841e-4cbc-83f5-565ac6b5dfcc-kube-api-access-6x79j\") pod \"volume-data-source-validator-5d456fdbcd-2tpsp\" (UID: \"f8a79e10-841e-4cbc-83f5-565ac6b5dfcc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2tpsp" Jul 09 03:42:45.887686 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.887618 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wl9b\" (UniqueName: \"kubernetes.io/projected/5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87-kube-api-access-9wl9b\") pod \"ingress-canary-b64b4\" (UID: \"5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87\") " pod="openshift-ingress-canary/ingress-canary-b64b4" Jul 09 03:42:45.887686 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.887668 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88d0565d-8fda-4829-833e-43b2b5e36610-registry-certificates\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.887686 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.887676 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2c564d7-7702-4488-8faa-a7c27f1608c0-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-tmxz8\" (UID: \"f2c564d7-7702-4488-8faa-a7c27f1608c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" Jul 09 03:42:45.887911 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.887728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a92ef34e-50ec-4d34-8d0f-a6e307d7a949-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-lbz6h\" (UID: \"a92ef34e-50ec-4d34-8d0f-a6e307d7a949\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" Jul 09 03:42:45.887911 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.887762 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79bfc\" (UniqueName: \"kubernetes.io/projected/c09b25d6-0ab2-4faa-8ead-f65a7e0a247a-kube-api-access-79bfc\") pod \"network-check-source-989f8b8dc-vh66t\" (UID: \"c09b25d6-0ab2-4faa-8ead-f65a7e0a247a\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vh66t" Jul 09 03:42:45.888340 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.888313 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2c564d7-7702-4488-8faa-a7c27f1608c0-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-tmxz8\" (UID: \"f2c564d7-7702-4488-8faa-a7c27f1608c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" Jul 09 03:42:45.889233 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.889205 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71b0a743-fa60-46e6-9fde-965caa5bdf86-tmp\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.890166 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.890142 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88d0565d-8fda-4829-833e-43b2b5e36610-image-registry-private-configuration\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.890883 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.890717 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f2c564d7-7702-4488-8faa-a7c27f1608c0-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-tmxz8\" (UID: \"f2c564d7-7702-4488-8faa-a7c27f1608c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" Jul 09 03:42:45.891288 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.891257 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88d0565d-8fda-4829-833e-43b2b5e36610-trusted-ca\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.891975 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.891954 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-stats-auth\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:45.896426 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.895802 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcmxb\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-kube-api-access-qcmxb\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.896426 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.896313 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-default-certificate\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:45.897414 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.897387 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71b0a743-fa60-46e6-9fde-965caa5bdf86-serving-cert\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.897556 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.897430 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a92ef34e-50ec-4d34-8d0f-a6e307d7a949-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-lbz6h\" (UID: \"a92ef34e-50ec-4d34-8d0f-a6e307d7a949\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" Jul 09 03:42:45.898475 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.898053 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-bound-sa-token\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.898475 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.898143 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gngd\" (UniqueName: \"kubernetes.io/projected/71b0a743-fa60-46e6-9fde-965caa5bdf86-kube-api-access-5gngd\") pod \"insights-operator-5c847dcfc4-ddhcf\" (UID: \"71b0a743-fa60-46e6-9fde-965caa5bdf86\") " pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.899043 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.898990 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwws2\" (UniqueName: \"kubernetes.io/projected/93f8e14e-aea8-496e-b728-1554938d338a-kube-api-access-hwws2\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:45.899234 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.899216 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stspc\" (UniqueName: \"kubernetes.io/projected/a92ef34e-50ec-4d34-8d0f-a6e307d7a949-kube-api-access-stspc\") pod \"service-ca-operator-5d46d9ffd4-lbz6h\" (UID: \"a92ef34e-50ec-4d34-8d0f-a6e307d7a949\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" Jul 09 03:42:45.900582 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.900540 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7993763-77e3-47b3-a32d-aa4b7c406413-serving-cert\") pod \"console-operator-68b757fb67-gskrv\" (UID: \"d7993763-77e3-47b3-a32d-aa4b7c406413\") " pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:42:45.901094 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.901069 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zgrv\" (UniqueName: \"kubernetes.io/projected/f2c564d7-7702-4488-8faa-a7c27f1608c0-kube-api-access-6zgrv\") pod \"kube-storage-version-migrator-operator-846bf668f7-tmxz8\" (UID: \"f2c564d7-7702-4488-8faa-a7c27f1608c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" Jul 09 03:42:45.901332 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.901245 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ljdr\" (UniqueName: \"kubernetes.io/projected/d7993763-77e3-47b3-a32d-aa4b7c406413-kube-api-access-4ljdr\") pod \"console-operator-68b757fb67-gskrv\" (UID: \"d7993763-77e3-47b3-a32d-aa4b7c406413\") " pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:42:45.902465 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.902444 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88d0565d-8fda-4829-833e-43b2b5e36610-installation-pull-secrets\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:45.902777 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.902758 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl4q5\" (UniqueName: \"kubernetes.io/projected/d01b29b9-9680-4c59-acad-460f17b275f4-kube-api-access-sl4q5\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:42:45.903436 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.903414 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tldng\" (UniqueName: \"kubernetes.io/projected/27fcabd6-d124-4abd-8b2d-86cc903a5693-kube-api-access-tldng\") pod \"cluster-samples-operator-56fcf86bf-s29mn\" (UID: \"27fcabd6-d124-4abd-8b2d-86cc903a5693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" Jul 09 03:42:45.903760 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.903743 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x79j\" (UniqueName: \"kubernetes.io/projected/f8a79e10-841e-4cbc-83f5-565ac6b5dfcc-kube-api-access-6x79j\") pod \"volume-data-source-validator-5d456fdbcd-2tpsp\" (UID: \"f8a79e10-841e-4cbc-83f5-565ac6b5dfcc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2tpsp" Jul 09 03:42:45.988522 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.988485 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wl9b\" (UniqueName: \"kubernetes.io/projected/5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87-kube-api-access-9wl9b\") pod \"ingress-canary-b64b4\" (UID: \"5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87\") " pod="openshift-ingress-canary/ingress-canary-b64b4" Jul 09 03:42:45.988738 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.988532 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79bfc\" (UniqueName: \"kubernetes.io/projected/c09b25d6-0ab2-4faa-8ead-f65a7e0a247a-kube-api-access-79bfc\") pod \"network-check-source-989f8b8dc-vh66t\" (UID: \"c09b25d6-0ab2-4faa-8ead-f65a7e0a247a\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vh66t" Jul 09 03:42:45.988738 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.988583 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64ca5948-6243-4377-9c91-a3da567aff5e-config-volume\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:45.988738 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.988610 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnczj\" (UniqueName: \"kubernetes.io/projected/64ca5948-6243-4377-9c91-a3da567aff5e-kube-api-access-nnczj\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:45.988738 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.988677 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64ca5948-6243-4377-9c91-a3da567aff5e-tmp-dir\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:45.988738 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.988704 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:45.988738 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.988731 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87-cert\") pod \"ingress-canary-b64b4\" (UID: \"5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87\") " pod="openshift-ingress-canary/ingress-canary-b64b4" Jul 09 03:42:45.989036 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:45.988854 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 03:42:45.989036 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:45.988921 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls podName:64ca5948-6243-4377-9c91-a3da567aff5e nodeName:}" failed. No retries permitted until 2026-07-09 03:42:46.488900319 +0000 UTC m=+37.120560388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls") pod "dns-default-mzcvn" (UID: "64ca5948-6243-4377-9c91-a3da567aff5e") : secret "dns-default-metrics-tls" not found Jul 09 03:42:45.989145 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.989060 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64ca5948-6243-4377-9c91-a3da567aff5e-tmp-dir\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:45.989240 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.989221 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64ca5948-6243-4377-9c91-a3da567aff5e-config-volume\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:45.991243 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.991221 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87-cert\") pod \"ingress-canary-b64b4\" (UID: \"5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87\") " pod="openshift-ingress-canary/ingress-canary-b64b4" Jul 09 03:42:45.995241 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.995197 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" Jul 09 03:42:45.997812 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.997783 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnczj\" (UniqueName: \"kubernetes.io/projected/64ca5948-6243-4377-9c91-a3da567aff5e-kube-api-access-nnczj\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:45.998197 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.998176 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79bfc\" (UniqueName: \"kubernetes.io/projected/c09b25d6-0ab2-4faa-8ead-f65a7e0a247a-kube-api-access-79bfc\") pod \"network-check-source-989f8b8dc-vh66t\" (UID: \"c09b25d6-0ab2-4faa-8ead-f65a7e0a247a\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vh66t" Jul 09 03:42:45.998273 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:45.998246 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wl9b\" (UniqueName: \"kubernetes.io/projected/5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87-kube-api-access-9wl9b\") pod \"ingress-canary-b64b4\" (UID: \"5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87\") " pod="openshift-ingress-canary/ingress-canary-b64b4" Jul 09 03:42:46.026622 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.026588 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2tpsp" Jul 09 03:42:46.050063 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.050026 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:42:46.065658 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.065608 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" Jul 09 03:42:46.072505 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.072471 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" Jul 09 03:42:46.081022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.080941 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vh66t" Jul 09 03:42:46.087388 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.087220 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b64b4" Jul 09 03:42:46.147722 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.147686 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ddhcf"] Jul 09 03:42:46.392534 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.392493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:46.392748 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.392604 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:46.392748 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.392659 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:46.392748 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:46.392664 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 03:42:46.392748 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:46.392687 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f8c949fc8-jw5ks: secret "image-registry-tls" not found Jul 09 03:42:46.392748 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.392719 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:42:46.392987 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:46.392755 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls podName:88d0565d-8fda-4829-833e-43b2b5e36610 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:47.392735254 +0000 UTC m=+38.024395335 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls") pod "image-registry-7f8c949fc8-jw5ks" (UID: "88d0565d-8fda-4829-833e-43b2b5e36610") : secret "image-registry-tls" not found Jul 09 03:42:46.392987 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.392781 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s29mn\" (UID: \"27fcabd6-d124-4abd-8b2d-86cc903a5693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" Jul 09 03:42:46.392987 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:46.392812 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 03:42:46.392987 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:46.392817 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 03:42:46.392987 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:46.392834 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle podName:93f8e14e-aea8-496e-b728-1554938d338a nodeName:}" failed. No retries permitted until 2026-07-09 03:42:47.392822762 +0000 UTC m=+38.024482827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle") pod "router-default-7dfb6bc4d7-d7vz4" (UID: "93f8e14e-aea8-496e-b728-1554938d338a") : configmap references non-existent config key: service-ca.crt Jul 09 03:42:46.392987 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:46.392854 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs podName:93f8e14e-aea8-496e-b728-1554938d338a nodeName:}" failed. No retries permitted until 2026-07-09 03:42:47.392842609 +0000 UTC m=+38.024502695 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs") pod "router-default-7dfb6bc4d7-d7vz4" (UID: "93f8e14e-aea8-496e-b728-1554938d338a") : secret "router-metrics-certs-default" not found Jul 09 03:42:46.392987 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:46.392873 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls podName:d01b29b9-9680-4c59-acad-460f17b275f4 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:47.392862828 +0000 UTC m=+38.024522894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tl5sq" (UID: "d01b29b9-9680-4c59-acad-460f17b275f4") : secret "cluster-monitoring-operator-tls" not found Jul 09 03:42:46.392987 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:46.392896 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 03:42:46.392987 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:46.392934 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls podName:27fcabd6-d124-4abd-8b2d-86cc903a5693 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:47.392924259 +0000 UTC m=+38.024584322 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-s29mn" (UID: "27fcabd6-d124-4abd-8b2d-86cc903a5693") : secret "samples-operator-tls" not found Jul 09 03:42:46.493943 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.493855 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:46.494099 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:46.494041 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 03:42:46.494154 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:46.494128 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls podName:64ca5948-6243-4377-9c91-a3da567aff5e nodeName:}" failed. No retries permitted until 2026-07-09 03:42:47.494105076 +0000 UTC m=+38.125765145 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls") pod "dns-default-mzcvn" (UID: "64ca5948-6243-4377-9c91-a3da567aff5e") : secret "dns-default-metrics-tls" not found Jul 09 03:42:46.859374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.859291 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:42:46.859561 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.859291 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:42:46.859561 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.859292 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:42:46.863578 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.863552 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 03:42:46.863578 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.863552 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-54tq2\"" Jul 09 03:42:46.863800 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.863552 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ch8mv\"" Jul 09 03:42:46.863800 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:46.863625 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 03:42:47.403861 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:47.403825 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:47.404733 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:47.403880 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:47.404733 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:47.403940 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:42:47.404733 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:47.403970 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s29mn\" (UID: \"27fcabd6-d124-4abd-8b2d-86cc903a5693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" Jul 09 03:42:47.404733 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:47.404005 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:47.404733 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:47.404028 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle podName:93f8e14e-aea8-496e-b728-1554938d338a nodeName:}" failed. No retries permitted until 2026-07-09 03:42:49.404003563 +0000 UTC m=+40.035663652 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle") pod "router-default-7dfb6bc4d7-d7vz4" (UID: "93f8e14e-aea8-496e-b728-1554938d338a") : configmap references non-existent config key: service-ca.crt Jul 09 03:42:47.404733 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:47.404045 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 03:42:47.404733 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:47.404081 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 03:42:47.404733 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:47.404090 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 03:42:47.404733 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:47.404103 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs podName:93f8e14e-aea8-496e-b728-1554938d338a nodeName:}" failed. No retries permitted until 2026-07-09 03:42:49.404086425 +0000 UTC m=+40.035746492 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs") pod "router-default-7dfb6bc4d7-d7vz4" (UID: "93f8e14e-aea8-496e-b728-1554938d338a") : secret "router-metrics-certs-default" not found Jul 09 03:42:47.404733 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:47.404092 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f8c949fc8-jw5ks: secret "image-registry-tls" not found Jul 09 03:42:47.404733 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:47.404133 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls podName:d01b29b9-9680-4c59-acad-460f17b275f4 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:49.404122079 +0000 UTC m=+40.035782160 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tl5sq" (UID: "d01b29b9-9680-4c59-acad-460f17b275f4") : secret "cluster-monitoring-operator-tls" not found Jul 09 03:42:47.404733 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:47.404138 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 03:42:47.404733 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:47.404153 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls podName:88d0565d-8fda-4829-833e-43b2b5e36610 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:49.404144633 +0000 UTC m=+40.035804697 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls") pod "image-registry-7f8c949fc8-jw5ks" (UID: "88d0565d-8fda-4829-833e-43b2b5e36610") : secret "image-registry-tls" not found Jul 09 03:42:47.404733 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:47.404175 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls podName:27fcabd6-d124-4abd-8b2d-86cc903a5693 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:49.404164309 +0000 UTC m=+40.035824374 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-s29mn" (UID: "27fcabd6-d124-4abd-8b2d-86cc903a5693") : secret "samples-operator-tls" not found Jul 09 03:42:47.505318 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:47.505281 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:47.505562 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:47.505443 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 03:42:47.505562 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:47.505510 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls podName:64ca5948-6243-4377-9c91-a3da567aff5e nodeName:}" failed. No retries permitted until 2026-07-09 03:42:49.505493742 +0000 UTC m=+40.137153816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls") pod "dns-default-mzcvn" (UID: "64ca5948-6243-4377-9c91-a3da567aff5e") : secret "dns-default-metrics-tls" not found Jul 09 03:42:47.596737 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:47.596696 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b0a743_fa60_46e6_9fde_965caa5bdf86.slice/crio-920547f304f8a782988f6afd3d86c85aecfc495a63950cd532066ed51f56199f WatchSource:0}: Error finding container 920547f304f8a782988f6afd3d86c85aecfc495a63950cd532066ed51f56199f: Status 404 returned error can't find the container with id 920547f304f8a782988f6afd3d86c85aecfc495a63950cd532066ed51f56199f Jul 09 03:42:47.825897 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:47.825864 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-vh66t"] Jul 09 03:42:47.832204 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:47.832159 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-gskrv"] Jul 09 03:42:47.842594 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:47.842567 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h"] Jul 09 03:42:47.845465 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:47.845442 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8"] Jul 09 03:42:47.854387 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:47.854251 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2tpsp"] Jul 09 03:42:47.857455 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:47.857421 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b64b4"] Jul 09 03:42:47.859911 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:47.859874 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a79e10_841e_4cbc_83f5_565ac6b5dfcc.slice/crio-09715ebcdff31e41f2176a706fffa20c345b2628b7201710220dcd23a097d107 WatchSource:0}: Error finding container 09715ebcdff31e41f2176a706fffa20c345b2628b7201710220dcd23a097d107: Status 404 returned error can't find the container with id 09715ebcdff31e41f2176a706fffa20c345b2628b7201710220dcd23a097d107 Jul 09 03:42:47.860296 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:47.860257 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e6ba6a4_190b_4e10_888b_b9f0ed2f0a87.slice/crio-7dc3344a075c7d2d414fc6cc21518d6cc6cc7e16d1536e682d2fc036be46aa63 WatchSource:0}: Error finding container 7dc3344a075c7d2d414fc6cc21518d6cc6cc7e16d1536e682d2fc036be46aa63: Status 404 returned error can't find the container with id 7dc3344a075c7d2d414fc6cc21518d6cc6cc7e16d1536e682d2fc036be46aa63 Jul 09 03:42:48.007679 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:48.007639 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkpv2" event={"ID":"e0abbca0-fa5c-40f8-be6d-225daabb1d4e","Type":"ContainerStarted","Data":"68af22a6659dde9d14bcfa328f8563cfd4492a82acbb1a959b730c308aef86a3"} Jul 09 03:42:48.008849 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:48.008820 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" event={"ID":"71b0a743-fa60-46e6-9fde-965caa5bdf86","Type":"ContainerStarted","Data":"920547f304f8a782988f6afd3d86c85aecfc495a63950cd532066ed51f56199f"} Jul 09 03:42:48.010052 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:48.010025 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" event={"ID":"a92ef34e-50ec-4d34-8d0f-a6e307d7a949","Type":"ContainerStarted","Data":"089d88a7bd3c069f2c071e8198f94378b80714654331d076d733d3a08fa6b8ec"} Jul 09 03:42:48.011576 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:48.011541 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-gskrv" event={"ID":"d7993763-77e3-47b3-a32d-aa4b7c406413","Type":"ContainerStarted","Data":"2221c5358f172e4643080d2dc0ea0f732e5b5a31f1cc4e277979bdf5a52790dc"} Jul 09 03:42:48.012878 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:48.012845 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2tpsp" event={"ID":"f8a79e10-841e-4cbc-83f5-565ac6b5dfcc","Type":"ContainerStarted","Data":"09715ebcdff31e41f2176a706fffa20c345b2628b7201710220dcd23a097d107"} Jul 09 03:42:48.014004 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:48.013977 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vh66t" event={"ID":"c09b25d6-0ab2-4faa-8ead-f65a7e0a247a","Type":"ContainerStarted","Data":"49dad59176692a022cd0f3f93bc62bfd4367014e4727aac53650acf0a79c4d61"} Jul 09 03:42:48.015270 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:48.015242 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b64b4" event={"ID":"5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87","Type":"ContainerStarted","Data":"7dc3344a075c7d2d414fc6cc21518d6cc6cc7e16d1536e682d2fc036be46aa63"} Jul 09 03:42:48.016568 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:48.016535 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" event={"ID":"f2c564d7-7702-4488-8faa-a7c27f1608c0","Type":"ContainerStarted","Data":"2fbfb34cd3778ee6464c0d703f4e625e8ea1e3361af31b7f8a9aa10a7e1bff16"} Jul 09 03:42:49.029554 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:49.029512 2566 generic.go:356] "Generic (PLEG): container finished" podID="e0abbca0-fa5c-40f8-be6d-225daabb1d4e" containerID="68af22a6659dde9d14bcfa328f8563cfd4492a82acbb1a959b730c308aef86a3" exitCode=0 Jul 09 03:42:49.030045 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:49.029582 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkpv2" event={"ID":"e0abbca0-fa5c-40f8-be6d-225daabb1d4e","Type":"ContainerDied","Data":"68af22a6659dde9d14bcfa328f8563cfd4492a82acbb1a959b730c308aef86a3"} Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:49.425598 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:49.425677 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:49.425749 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:49.425786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s29mn\" (UID: \"27fcabd6-d124-4abd-8b2d-86cc903a5693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:49.425825 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:49.425979 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:49.425996 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f8c949fc8-jw5ks: secret "image-registry-tls" not found Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:49.426056 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls podName:88d0565d-8fda-4829-833e-43b2b5e36610 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:53.426036543 +0000 UTC m=+44.057696621 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls") pod "image-registry-7f8c949fc8-jw5ks" (UID: "88d0565d-8fda-4829-833e-43b2b5e36610") : secret "image-registry-tls" not found Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:49.426128 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle podName:93f8e14e-aea8-496e-b728-1554938d338a nodeName:}" failed. No retries permitted until 2026-07-09 03:42:53.426118358 +0000 UTC m=+44.057778429 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle") pod "router-default-7dfb6bc4d7-d7vz4" (UID: "93f8e14e-aea8-496e-b728-1554938d338a") : configmap references non-existent config key: service-ca.crt Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:49.426188 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:49.426219 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs podName:93f8e14e-aea8-496e-b728-1554938d338a nodeName:}" failed. No retries permitted until 2026-07-09 03:42:53.426209125 +0000 UTC m=+44.057869191 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs") pod "router-default-7dfb6bc4d7-d7vz4" (UID: "93f8e14e-aea8-496e-b728-1554938d338a") : secret "router-metrics-certs-default" not found Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:49.426270 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:49.426304 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls podName:d01b29b9-9680-4c59-acad-460f17b275f4 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:53.426294058 +0000 UTC m=+44.057954128 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tl5sq" (UID: "d01b29b9-9680-4c59-acad-460f17b275f4") : secret "cluster-monitoring-operator-tls" not found Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:49.426359 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 03:42:49.426741 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:49.426390 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls podName:27fcabd6-d124-4abd-8b2d-86cc903a5693 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:53.426380115 +0000 UTC m=+44.058040178 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-s29mn" (UID: "27fcabd6-d124-4abd-8b2d-86cc903a5693") : secret "samples-operator-tls" not found Jul 09 03:42:49.526531 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:49.526493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:49.526832 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:49.526814 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 03:42:49.526923 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:49.526883 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls podName:64ca5948-6243-4377-9c91-a3da567aff5e nodeName:}" failed. No retries permitted until 2026-07-09 03:42:53.526862835 +0000 UTC m=+44.158522915 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls") pod "dns-default-mzcvn" (UID: "64ca5948-6243-4377-9c91-a3da567aff5e") : secret "dns-default-metrics-tls" not found Jul 09 03:42:50.040807 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:50.040691 2566 generic.go:356] "Generic (PLEG): container finished" podID="e0abbca0-fa5c-40f8-be6d-225daabb1d4e" containerID="78a3bffd957ae497810627ed82286d5b474d96cdd0a23cb6e946553cfc161ad6" exitCode=0 Jul 09 03:42:50.041279 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:50.040899 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkpv2" event={"ID":"e0abbca0-fa5c-40f8-be6d-225daabb1d4e","Type":"ContainerDied","Data":"78a3bffd957ae497810627ed82286d5b474d96cdd0a23cb6e946553cfc161ad6"} Jul 09 03:42:50.553539 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:50.553477 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:50.633670 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:50.633622 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:53.462831 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:53.462783 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:42:53.463289 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:53.462923 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:53.463289 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:53.462947 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 03:42:53.463289 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:53.462971 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f8c949fc8-jw5ks: secret "image-registry-tls" not found Jul 09 03:42:53.463289 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:53.463023 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 03:42:53.463289 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:53.463032 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls podName:88d0565d-8fda-4829-833e-43b2b5e36610 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:01.463015087 +0000 UTC m=+52.094675154 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls") pod "image-registry-7f8c949fc8-jw5ks" (UID: "88d0565d-8fda-4829-833e-43b2b5e36610") : secret "image-registry-tls" not found Jul 09 03:42:53.463289 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:53.462950 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:42:53.463289 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:53.463079 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs podName:93f8e14e-aea8-496e-b728-1554938d338a nodeName:}" failed. No retries permitted until 2026-07-09 03:43:01.463061429 +0000 UTC m=+52.094721495 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs") pod "router-default-7dfb6bc4d7-d7vz4" (UID: "93f8e14e-aea8-496e-b728-1554938d338a") : secret "router-metrics-certs-default" not found Jul 09 03:42:53.463289 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:53.463112 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle podName:93f8e14e-aea8-496e-b728-1554938d338a nodeName:}" failed. No retries permitted until 2026-07-09 03:43:01.463102578 +0000 UTC m=+52.094762642 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle") pod "router-default-7dfb6bc4d7-d7vz4" (UID: "93f8e14e-aea8-496e-b728-1554938d338a") : configmap references non-existent config key: service-ca.crt Jul 09 03:42:53.463289 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:53.463171 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:42:53.463289 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:53.463197 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s29mn\" (UID: \"27fcabd6-d124-4abd-8b2d-86cc903a5693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" Jul 09 03:42:53.463289 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:53.463301 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 03:42:53.463778 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:53.463325 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 03:42:53.463778 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:53.463343 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls podName:27fcabd6-d124-4abd-8b2d-86cc903a5693 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:01.463333627 +0000 UTC m=+52.094993691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-s29mn" (UID: "27fcabd6-d124-4abd-8b2d-86cc903a5693") : secret "samples-operator-tls" not found Jul 09 03:42:53.463778 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:53.463391 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls podName:d01b29b9-9680-4c59-acad-460f17b275f4 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:01.463374023 +0000 UTC m=+52.095034105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tl5sq" (UID: "d01b29b9-9680-4c59-acad-460f17b275f4") : secret "cluster-monitoring-operator-tls" not found Jul 09 03:42:53.563864 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:53.563826 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:42:53.564060 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:53.564006 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 03:42:53.564125 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:53.564100 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls podName:64ca5948-6243-4377-9c91-a3da567aff5e nodeName:}" failed. No retries permitted until 2026-07-09 03:43:01.564077391 +0000 UTC m=+52.195737472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls") pod "dns-default-mzcvn" (UID: "64ca5948-6243-4377-9c91-a3da567aff5e") : secret "dns-default-metrics-tls" not found Jul 09 03:42:54.087375 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.087338 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc"] Jul 09 03:42:54.106680 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.106632 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc"] Jul 09 03:42:54.106834 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.106749 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" Jul 09 03:42:54.109420 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.109395 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 09 03:42:54.109542 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.109404 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-72q9v\"" Jul 09 03:42:54.109542 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.109530 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 09 03:42:54.209753 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.209716 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:54.263981 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.263946 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:54.270245 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.270219 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c76314eb-b801-418f-9800-c7b0f42c0f97-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xpkvc\" (UID: \"c76314eb-b801-418f-9800-c7b0f42c0f97\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" Jul 09 03:42:54.270368 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.270326 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xpkvc\" (UID: \"c76314eb-b801-418f-9800-c7b0f42c0f97\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" Jul 09 03:42:54.317858 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.317821 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:54.371449 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.371363 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xpkvc\" (UID: \"c76314eb-b801-418f-9800-c7b0f42c0f97\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" Jul 09 03:42:54.371669 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.371508 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c76314eb-b801-418f-9800-c7b0f42c0f97-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xpkvc\" (UID: \"c76314eb-b801-418f-9800-c7b0f42c0f97\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" Jul 09 03:42:54.371669 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:54.371559 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 03:42:54.371669 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:54.371640 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert podName:c76314eb-b801-418f-9800-c7b0f42c0f97 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:54.871620512 +0000 UTC m=+45.503280592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xpkvc" (UID: "c76314eb-b801-418f-9800-c7b0f42c0f97") : secret "networking-console-plugin-cert" not found Jul 09 03:42:54.372338 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.372314 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c76314eb-b801-418f-9800-c7b0f42c0f97-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xpkvc\" (UID: \"c76314eb-b801-418f-9800-c7b0f42c0f97\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" Jul 09 03:42:54.376138 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.376106 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:54.438599 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.438558 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:54.876739 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:54.876696 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xpkvc\" (UID: \"c76314eb-b801-418f-9800-c7b0f42c0f97\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" Jul 09 03:42:54.877208 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:54.876887 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 03:42:54.877208 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:54.876969 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert podName:c76314eb-b801-418f-9800-c7b0f42c0f97 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:55.876946158 +0000 UTC m=+46.508606239 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xpkvc" (UID: "c76314eb-b801-418f-9800-c7b0f42c0f97") : secret "networking-console-plugin-cert" not found Jul 09 03:42:55.885683 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:55.885624 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xpkvc\" (UID: \"c76314eb-b801-418f-9800-c7b0f42c0f97\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" Jul 09 03:42:55.886191 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:55.885772 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 03:42:55.886191 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:55.885848 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert podName:c76314eb-b801-418f-9800-c7b0f42c0f97 nodeName:}" failed. No retries permitted until 2026-07-09 03:42:57.885823967 +0000 UTC m=+48.517484032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xpkvc" (UID: "c76314eb-b801-418f-9800-c7b0f42c0f97") : secret "networking-console-plugin-cert" not found Jul 09 03:42:56.063481 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.063348 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" event={"ID":"a92ef34e-50ec-4d34-8d0f-a6e307d7a949","Type":"ContainerStarted","Data":"d8a28a5c9e9a3624e92afafdd9c75a5ae07bca549a383af57da13a14cea12095"} Jul 09 03:42:56.066492 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.066465 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/0.log" Jul 09 03:42:56.066690 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.066512 2566 generic.go:356] "Generic (PLEG): container finished" podID="d7993763-77e3-47b3-a32d-aa4b7c406413" containerID="8e333223ab4a29c2a5bb1547e481e3fec27a8e8c3d2e0600fbb6bef09254a05d" exitCode=255 Jul 09 03:42:56.066828 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.066799 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-gskrv" event={"ID":"d7993763-77e3-47b3-a32d-aa4b7c406413","Type":"ContainerDied","Data":"8e333223ab4a29c2a5bb1547e481e3fec27a8e8c3d2e0600fbb6bef09254a05d"} Jul 09 03:42:56.067662 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.066933 2566 scope.go:117] "RemoveContainer" containerID="8e333223ab4a29c2a5bb1547e481e3fec27a8e8c3d2e0600fbb6bef09254a05d" Jul 09 03:42:56.071670 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.071625 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2tpsp" event={"ID":"f8a79e10-841e-4cbc-83f5-565ac6b5dfcc","Type":"ContainerStarted","Data":"e951875f00045b506f9737a9fa2029fc6719d844c39996ee9c671195f47acb95"} Jul 09 03:42:56.074157 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.073769 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vh66t" event={"ID":"c09b25d6-0ab2-4faa-8ead-f65a7e0a247a","Type":"ContainerStarted","Data":"3440cfdb50c2e1b56c9a34b0d835774074ccafbd7d385a4083be6e73ba6ab8c0"} Jul 09 03:42:56.075328 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.075301 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b64b4" event={"ID":"5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87","Type":"ContainerStarted","Data":"3fb49618ed3a5336a179b4026a18bebe998381f6c2ff9ed8c7a3b7a7cdf81853"} Jul 09 03:42:56.077261 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.077211 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" event={"ID":"f2c564d7-7702-4488-8faa-a7c27f1608c0","Type":"ContainerStarted","Data":"f346cae08cb0f5feee07bab178eb49ab8c103e07acc186eddbebeee6fd7f79cb"} Jul 09 03:42:56.080845 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.080794 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" podStartSLOduration=20.275185098 podStartE2EDuration="28.080774205s" totalImagesPullingTime="7.805589107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:56.079517869 +0000 UTC m=+46.711177955" watchObservedRunningTime="2026-07-09 03:42:56.080774205 +0000 UTC m=+46.712434272" Jul 09 03:42:56.088366 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.088161 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkpv2" event={"ID":"e0abbca0-fa5c-40f8-be6d-225daabb1d4e","Type":"ContainerStarted","Data":"5fd6817199309b87e2207933c9eac368f5539956ed930dcd2f2b9297d1696e8f"} Jul 09 03:42:56.089985 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.089880 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" event={"ID":"71b0a743-fa60-46e6-9fde-965caa5bdf86","Type":"ContainerStarted","Data":"32ed727179044cda0f0518951db50245952e7fd3582f325341debc671be742a2"} Jul 09 03:42:56.125802 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.125737 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vh66t" podStartSLOduration=21.307281203 podStartE2EDuration="29.125713162s" totalImagesPullingTime="7.818431959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:56.124537063 +0000 UTC m=+46.756197152" watchObservedRunningTime="2026-07-09 03:42:56.125713162 +0000 UTC m=+46.757373249" Jul 09 03:42:56.162089 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.161910 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b64b4" podStartSLOduration=3.356525798 podStartE2EDuration="11.161886198s" totalImagesPullingTime="7.8053604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:56.141865129 +0000 UTC m=+46.773525217" watchObservedRunningTime="2026-07-09 03:42:56.161886198 +0000 UTC m=+46.793546286" Jul 09 03:42:56.181286 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.181179 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" podStartSLOduration=22.374611992 podStartE2EDuration="30.181156243s" totalImagesPullingTime="7.806544251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:56.180903306 +0000 UTC m=+46.812563393" watchObservedRunningTime="2026-07-09 03:42:56.181156243 +0000 UTC m=+46.812816331" Jul 09 03:42:56.182595 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.182556 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2tpsp" podStartSLOduration=27.399457454 podStartE2EDuration="35.182537453s" totalImagesPullingTime="7.783079999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:56.163624441 +0000 UTC m=+46.795284531" watchObservedRunningTime="2026-07-09 03:42:56.182537453 +0000 UTC m=+46.814197546" Jul 09 03:42:56.206491 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.206060 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tkpv2" podStartSLOduration=12.788281323 podStartE2EDuration="46.206037457s" totalImagesPullingTime="33.417756134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:42:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:56.205495883 +0000 UTC m=+46.837155968" watchObservedRunningTime="2026-07-09 03:42:56.206037457 +0000 UTC m=+46.837697543" Jul 09 03:42:56.222569 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.222520 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" podStartSLOduration=28.154249657 podStartE2EDuration="36.222507719s" totalImagesPullingTime="8.068258062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:42:56.221182184 +0000 UTC m=+46.852842282" watchObservedRunningTime="2026-07-09 03:42:56.222507719 +0000 UTC m=+46.854167804" Jul 09 03:42:56.335041 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.334956 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:56.435687 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.435572 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:56.482712 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:56.482675 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:57.095409 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.095383 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 03:42:57.096163 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.096136 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/0.log" Jul 09 03:42:57.096276 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.096182 2566 generic.go:356] "Generic (PLEG): container finished" podID="d7993763-77e3-47b3-a32d-aa4b7c406413" containerID="5313f92dc959b28f279cf18439af2720fb0610cca701441dc9fe3920ca9a7d44" exitCode=255 Jul 09 03:42:57.096992 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.096970 2566 scope.go:117] "RemoveContainer" containerID="5313f92dc959b28f279cf18439af2720fb0610cca701441dc9fe3920ca9a7d44" Jul 09 03:42:57.097125 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.097000 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-gskrv" event={"ID":"d7993763-77e3-47b3-a32d-aa4b7c406413","Type":"ContainerDied","Data":"5313f92dc959b28f279cf18439af2720fb0610cca701441dc9fe3920ca9a7d44"} Jul 09 03:42:57.097125 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.097095 2566 scope.go:117] "RemoveContainer" containerID="8e333223ab4a29c2a5bb1547e481e3fec27a8e8c3d2e0600fbb6bef09254a05d" Jul 09 03:42:57.097243 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:57.097134 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-gskrv_openshift-console-operator(d7993763-77e3-47b3-a32d-aa4b7c406413)\"" pod="openshift-console-operator/console-operator-68b757fb67-gskrv" podUID="d7993763-77e3-47b3-a32d-aa4b7c406413" Jul 09 03:42:57.364240 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.364148 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-55v58"] Jul 09 03:42:57.395344 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.395311 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-55v58"] Jul 09 03:42:57.395533 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.395422 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:57.398140 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.398114 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 03:42:57.398285 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.398183 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 03:42:57.398353 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.398281 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-phtqr\"" Jul 09 03:42:57.419594 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.419557 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:57.502709 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.502675 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/210d8dd5-d21a-4349-92ee-2f55b6a0786c-data-volume\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:57.502893 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.502745 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:57.502893 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.502788 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/210d8dd5-d21a-4349-92ee-2f55b6a0786c-crio-socket\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:57.502893 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.502818 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:57.503086 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.502904 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkjtl\" (UniqueName: \"kubernetes.io/projected/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-api-access-mkjtl\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:57.536772 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.536737 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:57.603849 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.603812 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkjtl\" (UniqueName: \"kubernetes.io/projected/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-api-access-mkjtl\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:57.604011 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.603921 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/210d8dd5-d21a-4349-92ee-2f55b6a0786c-data-volume\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:57.604011 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.603978 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:57.604099 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.604012 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/210d8dd5-d21a-4349-92ee-2f55b6a0786c-crio-socket\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:57.604099 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.604045 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:57.604169 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.604115 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/210d8dd5-d21a-4349-92ee-2f55b6a0786c-crio-socket\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:57.604169 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:57.604122 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 03:42:57.604237 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:57.604126 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 03:42:57.604237 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:57.604210 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls podName:210d8dd5-d21a-4349-92ee-2f55b6a0786c nodeName:}" failed. No retries permitted until 2026-07-09 03:42:58.104188446 +0000 UTC m=+48.735848529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-55v58" (UID: "210d8dd5-d21a-4349-92ee-2f55b6a0786c") : secret "insights-runtime-extractor-tls" not found Jul 09 03:42:57.604318 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:57.604261 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm podName:210d8dd5-d21a-4349-92ee-2f55b6a0786c nodeName:}" failed. No retries permitted until 2026-07-09 03:42:58.104244381 +0000 UTC m=+48.735904444 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-55v58" (UID: "210d8dd5-d21a-4349-92ee-2f55b6a0786c") : configmap "kube-rbac-proxy" not found Jul 09 03:42:57.604318 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.604301 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/210d8dd5-d21a-4349-92ee-2f55b6a0786c-data-volume\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:57.616105 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.616043 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkjtl\" (UniqueName: \"kubernetes.io/projected/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-api-access-mkjtl\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:57.643917 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.643883 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:57.744681 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.744639 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:57.906998 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:57.906963 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xpkvc\" (UID: \"c76314eb-b801-418f-9800-c7b0f42c0f97\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" Jul 09 03:42:57.907179 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:57.907113 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 03:42:57.907235 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:57.907183 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert podName:c76314eb-b801-418f-9800-c7b0f42c0f97 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:01.90716767 +0000 UTC m=+52.538827734 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xpkvc" (UID: "c76314eb-b801-418f-9800-c7b0f42c0f97") : secret "networking-console-plugin-cert" not found Jul 09 03:42:58.079883 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.079846 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:58.100597 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.100574 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 03:42:58.101046 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.100992 2566 scope.go:117] "RemoveContainer" containerID="5313f92dc959b28f279cf18439af2720fb0610cca701441dc9fe3920ca9a7d44" Jul 09 03:42:58.101213 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:58.101190 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-gskrv_openshift-console-operator(d7993763-77e3-47b3-a32d-aa4b7c406413)\"" pod="openshift-console-operator/console-operator-68b757fb67-gskrv" podUID="d7993763-77e3-47b3-a32d-aa4b7c406413" Jul 09 03:42:58.108690 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.108669 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:58.108799 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.108709 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:58.108837 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:58.108796 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 03:42:58.108837 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:58.108813 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 03:42:58.108897 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:58.108851 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls podName:210d8dd5-d21a-4349-92ee-2f55b6a0786c nodeName:}" failed. No retries permitted until 2026-07-09 03:42:59.108835186 +0000 UTC m=+49.740495250 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-55v58" (UID: "210d8dd5-d21a-4349-92ee-2f55b6a0786c") : secret "insights-runtime-extractor-tls" not found Jul 09 03:42:58.108897 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:58.108865 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm podName:210d8dd5-d21a-4349-92ee-2f55b6a0786c nodeName:}" failed. No retries permitted until 2026-07-09 03:42:59.108858058 +0000 UTC m=+49.740518122 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-55v58" (UID: "210d8dd5-d21a-4349-92ee-2f55b6a0786c") : configmap "kube-rbac-proxy" not found Jul 09 03:42:58.180879 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.180796 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:58.347874 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.347846 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mf9s5_6ac5e518-8402-4297-8ce7-4f083d9c2df2/dns-node-resolver/0.log" Jul 09 03:42:58.352851 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.352822 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:58.404987 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.404955 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-f842c"] Jul 09 03:42:58.423238 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.423193 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:58.436467 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.436387 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-f842c"] Jul 09 03:42:58.436601 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.436515 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f842c" Jul 09 03:42:58.439247 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.439226 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-4t9qq\"" Jul 09 03:42:58.439390 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.439232 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 09 03:42:58.439390 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.439382 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 09 03:42:58.476518 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.476485 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:58.613612 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.613575 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv2mm\" (UniqueName: \"kubernetes.io/projected/898224dd-88b9-418a-9f84-7cc28899c820-kube-api-access-nv2mm\") pod \"migrator-d57d58cc9-f842c\" (UID: \"898224dd-88b9-418a-9f84-7cc28899c820\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f842c" Jul 09 03:42:58.714148 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.714073 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv2mm\" (UniqueName: \"kubernetes.io/projected/898224dd-88b9-418a-9f84-7cc28899c820-kube-api-access-nv2mm\") pod \"migrator-d57d58cc9-f842c\" (UID: \"898224dd-88b9-418a-9f84-7cc28899c820\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f842c" Jul 09 03:42:58.722411 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.722380 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv2mm\" (UniqueName: \"kubernetes.io/projected/898224dd-88b9-418a-9f84-7cc28899c820-kube-api-access-nv2mm\") pod \"migrator-d57d58cc9-f842c\" (UID: \"898224dd-88b9-418a-9f84-7cc28899c820\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f842c" Jul 09 03:42:58.746401 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.746372 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f842c" Jul 09 03:42:58.749785 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.749747 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:58.846770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.846734 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:42:58.883728 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:58.883701 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-f842c"] Jul 09 03:42:58.886638 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:42:58.886613 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod898224dd_88b9_418a_9f84_7cc28899c820.slice/crio-af5d92377f5f6fc77b1aa27fe122632ac30b9d285aa7ba121ba61baf16731605 WatchSource:0}: Error finding container af5d92377f5f6fc77b1aa27fe122632ac30b9d285aa7ba121ba61baf16731605: Status 404 returned error can't find the container with id af5d92377f5f6fc77b1aa27fe122632ac30b9d285aa7ba121ba61baf16731605 Jul 09 03:42:59.105311 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:59.105225 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f842c" event={"ID":"898224dd-88b9-418a-9f84-7cc28899c820","Type":"ContainerStarted","Data":"af5d92377f5f6fc77b1aa27fe122632ac30b9d285aa7ba121ba61baf16731605"} Jul 09 03:42:59.118885 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:59.118848 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:59.119026 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:59.118916 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:42:59.119026 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:59.118985 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 03:42:59.119026 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:59.119017 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 03:42:59.119131 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:59.119041 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls podName:210d8dd5-d21a-4349-92ee-2f55b6a0786c nodeName:}" failed. No retries permitted until 2026-07-09 03:43:01.119026264 +0000 UTC m=+51.750686328 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-55v58" (UID: "210d8dd5-d21a-4349-92ee-2f55b6a0786c") : secret "insights-runtime-extractor-tls" not found Jul 09 03:42:59.119131 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:42:59.119063 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm podName:210d8dd5-d21a-4349-92ee-2f55b6a0786c nodeName:}" failed. No retries permitted until 2026-07-09 03:43:01.119048247 +0000 UTC m=+51.750708317 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-55v58" (UID: "210d8dd5-d21a-4349-92ee-2f55b6a0786c") : configmap "kube-rbac-proxy" not found Jul 09 03:42:59.147089 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:59.147061 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6dggs_6da3145c-5f91-44ca-8465-ce298a4ee12e/node-ca/0.log" Jul 09 03:42:59.747413 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:59.747379 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b64b4_5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87/serve-healthcheck-canary/0.log" Jul 09 03:42:59.951418 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:59.951381 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-v7f99"] Jul 09 03:42:59.973498 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:59.973467 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-v7f99"] Jul 09 03:42:59.973720 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:59.973634 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-v7f99" Jul 09 03:42:59.976390 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:59.976361 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 09 03:42:59.976527 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:59.976398 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 09 03:42:59.976666 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:59.976632 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 09 03:42:59.977857 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:59.977832 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 09 03:42:59.977857 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:59.977847 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-v5lrk\"" Jul 09 03:42:59.994264 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:42:59.994123 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:00.045580 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:00.045500 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:00.129025 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:00.128995 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p69c\" (UniqueName: \"kubernetes.io/projected/85893533-f75e-41df-91c8-b33036f501fa-kube-api-access-8p69c\") pod \"service-ca-758f599c5-v7f99\" (UID: \"85893533-f75e-41df-91c8-b33036f501fa\") " pod="openshift-service-ca/service-ca-758f599c5-v7f99" Jul 09 03:43:00.129535 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:00.129067 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/85893533-f75e-41df-91c8-b33036f501fa-signing-key\") pod \"service-ca-758f599c5-v7f99\" (UID: \"85893533-f75e-41df-91c8-b33036f501fa\") " pod="openshift-service-ca/service-ca-758f599c5-v7f99" Jul 09 03:43:00.129535 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:00.129097 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/85893533-f75e-41df-91c8-b33036f501fa-signing-cabundle\") pod \"service-ca-758f599c5-v7f99\" (UID: \"85893533-f75e-41df-91c8-b33036f501fa\") " pod="openshift-service-ca/service-ca-758f599c5-v7f99" Jul 09 03:43:00.230480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:00.230444 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/85893533-f75e-41df-91c8-b33036f501fa-signing-key\") pod \"service-ca-758f599c5-v7f99\" (UID: \"85893533-f75e-41df-91c8-b33036f501fa\") " pod="openshift-service-ca/service-ca-758f599c5-v7f99" Jul 09 03:43:00.230480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:00.230482 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/85893533-f75e-41df-91c8-b33036f501fa-signing-cabundle\") pod \"service-ca-758f599c5-v7f99\" (UID: \"85893533-f75e-41df-91c8-b33036f501fa\") " pod="openshift-service-ca/service-ca-758f599c5-v7f99" Jul 09 03:43:00.230721 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:00.230684 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p69c\" (UniqueName: \"kubernetes.io/projected/85893533-f75e-41df-91c8-b33036f501fa-kube-api-access-8p69c\") pod \"service-ca-758f599c5-v7f99\" (UID: \"85893533-f75e-41df-91c8-b33036f501fa\") " pod="openshift-service-ca/service-ca-758f599c5-v7f99" Jul 09 03:43:00.231202 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:00.231178 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/85893533-f75e-41df-91c8-b33036f501fa-signing-cabundle\") pod \"service-ca-758f599c5-v7f99\" (UID: \"85893533-f75e-41df-91c8-b33036f501fa\") " pod="openshift-service-ca/service-ca-758f599c5-v7f99" Jul 09 03:43:00.232945 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:00.232922 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/85893533-f75e-41df-91c8-b33036f501fa-signing-key\") pod \"service-ca-758f599c5-v7f99\" (UID: \"85893533-f75e-41df-91c8-b33036f501fa\") " pod="openshift-service-ca/service-ca-758f599c5-v7f99" Jul 09 03:43:00.238196 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:00.238174 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p69c\" (UniqueName: \"kubernetes.io/projected/85893533-f75e-41df-91c8-b33036f501fa-kube-api-access-8p69c\") pod \"service-ca-758f599c5-v7f99\" (UID: \"85893533-f75e-41df-91c8-b33036f501fa\") " pod="openshift-service-ca/service-ca-758f599c5-v7f99" Jul 09 03:43:00.285224 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:00.285189 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-v7f99" Jul 09 03:43:00.646238 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:00.646213 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-v7f99"] Jul 09 03:43:00.710389 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:43:00.710341 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85893533_f75e_41df_91c8_b33036f501fa.slice/crio-304aab78d540eb22768bfe8134b8ab3922511da070dd336468138dafea0ba29b WatchSource:0}: Error finding container 304aab78d540eb22768bfe8134b8ab3922511da070dd336468138dafea0ba29b: Status 404 returned error can't find the container with id 304aab78d540eb22768bfe8134b8ab3922511da070dd336468138dafea0ba29b Jul 09 03:43:00.867448 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:00.867412 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:01.113849 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.113772 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f842c" event={"ID":"898224dd-88b9-418a-9f84-7cc28899c820","Type":"ContainerStarted","Data":"338211b986b2fab6aba4994983407fa4b2f3324f143d7cbf192eb9e346991b79"} Jul 09 03:43:01.113849 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.113823 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f842c" event={"ID":"898224dd-88b9-418a-9f84-7cc28899c820","Type":"ContainerStarted","Data":"8a20ad3810ec2a6588c6cb16ee3c21d291ee9147ac09b1362ffa0be00df03ff0"} Jul 09 03:43:01.115270 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.115245 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-v7f99" event={"ID":"85893533-f75e-41df-91c8-b33036f501fa","Type":"ContainerStarted","Data":"301ad520f438604f0eaa8833b4d19973a3d99781972525358b5eb93399388fcd"} Jul 09 03:43:01.115366 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.115289 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-v7f99" event={"ID":"85893533-f75e-41df-91c8-b33036f501fa","Type":"ContainerStarted","Data":"304aab78d540eb22768bfe8134b8ab3922511da070dd336468138dafea0ba29b"} Jul 09 03:43:01.133863 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.133814 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f842c" podStartSLOduration=1.294400499 podStartE2EDuration="3.133799703s" totalImagesPullingTime="1.839399204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:01.132664193 +0000 UTC m=+51.764324273" watchObservedRunningTime="2026-07-09 03:43:01.133799703 +0000 UTC m=+51.765459788" Jul 09 03:43:01.138059 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.138033 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:43:01.138153 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.138081 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:43:01.138224 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.138194 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 03:43:01.138274 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.138246 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 03:43:01.138274 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.138257 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls podName:210d8dd5-d21a-4349-92ee-2f55b6a0786c nodeName:}" failed. No retries permitted until 2026-07-09 03:43:05.138239411 +0000 UTC m=+55.769899492 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-55v58" (UID: "210d8dd5-d21a-4349-92ee-2f55b6a0786c") : secret "insights-runtime-extractor-tls" not found Jul 09 03:43:01.138383 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.138287 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm podName:210d8dd5-d21a-4349-92ee-2f55b6a0786c nodeName:}" failed. No retries permitted until 2026-07-09 03:43:05.138276015 +0000 UTC m=+55.769936079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-55v58" (UID: "210d8dd5-d21a-4349-92ee-2f55b6a0786c") : configmap "kube-rbac-proxy" not found Jul 09 03:43:01.153546 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.153505 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-v7f99" podStartSLOduration=2.153489933 podStartE2EDuration="2.153489933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:01.152788353 +0000 UTC m=+51.784448440" watchObservedRunningTime="2026-07-09 03:43:01.153489933 +0000 UTC m=+51.785150019" Jul 09 03:43:01.543362 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.543324 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:43:01.543581 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.543369 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:43:01.543581 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.543422 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:43:01.543581 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.543445 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s29mn\" (UID: \"27fcabd6-d124-4abd-8b2d-86cc903a5693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" Jul 09 03:43:01.543581 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.543491 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:43:01.543581 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.543527 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle podName:93f8e14e-aea8-496e-b728-1554938d338a nodeName:}" failed. No retries permitted until 2026-07-09 03:43:17.543508364 +0000 UTC m=+68.175168437 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle") pod "router-default-7dfb6bc4d7-d7vz4" (UID: "93f8e14e-aea8-496e-b728-1554938d338a") : configmap references non-existent config key: service-ca.crt Jul 09 03:43:01.543817 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.543589 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 03:43:01.543817 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.543594 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 03:43:01.543817 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.543611 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 03:43:01.543817 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.543625 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 03:43:01.543817 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.543643 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f8c949fc8-jw5ks: secret "image-registry-tls" not found Jul 09 03:43:01.543817 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.543661 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls podName:d01b29b9-9680-4c59-acad-460f17b275f4 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:17.543632485 +0000 UTC m=+68.175292571 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tl5sq" (UID: "d01b29b9-9680-4c59-acad-460f17b275f4") : secret "cluster-monitoring-operator-tls" not found Jul 09 03:43:01.543817 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.543683 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls podName:27fcabd6-d124-4abd-8b2d-86cc903a5693 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:17.543673404 +0000 UTC m=+68.175333473 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-s29mn" (UID: "27fcabd6-d124-4abd-8b2d-86cc903a5693") : secret "samples-operator-tls" not found Jul 09 03:43:01.543817 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.543701 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs podName:93f8e14e-aea8-496e-b728-1554938d338a nodeName:}" failed. No retries permitted until 2026-07-09 03:43:17.54369293 +0000 UTC m=+68.175352994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs") pod "router-default-7dfb6bc4d7-d7vz4" (UID: "93f8e14e-aea8-496e-b728-1554938d338a") : secret "router-metrics-certs-default" not found Jul 09 03:43:01.543817 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.543723 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls podName:88d0565d-8fda-4829-833e-43b2b5e36610 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:17.543714687 +0000 UTC m=+68.175374750 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls") pod "image-registry-7f8c949fc8-jw5ks" (UID: "88d0565d-8fda-4829-833e-43b2b5e36610") : secret "image-registry-tls" not found Jul 09 03:43:01.644448 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.644408 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:43:01.644617 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.644572 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 03:43:01.644693 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.644674 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls podName:64ca5948-6243-4377-9c91-a3da567aff5e nodeName:}" failed. No retries permitted until 2026-07-09 03:43:17.644634847 +0000 UTC m=+68.276294929 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls") pod "dns-default-mzcvn" (UID: "64ca5948-6243-4377-9c91-a3da567aff5e") : secret "dns-default-metrics-tls" not found Jul 09 03:43:01.649011 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.648983 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:01.947783 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:01.947742 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xpkvc\" (UID: \"c76314eb-b801-418f-9800-c7b0f42c0f97\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" Jul 09 03:43:01.947976 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.947921 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 03:43:01.948028 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:01.947988 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert podName:c76314eb-b801-418f-9800-c7b0f42c0f97 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:09.947972218 +0000 UTC m=+60.579632296 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xpkvc" (UID: "c76314eb-b801-418f-9800-c7b0f42c0f97") : secret "networking-console-plugin-cert" not found Jul 09 03:43:02.547482 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:02.547430 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:02.768129 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:02.768091 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:02.867494 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:02.867377 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:03.153760 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:03.153719 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:03.252927 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:03.252892 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:03.352992 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:03.352957 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:03.855400 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:03.855366 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:04.253382 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:04.253333 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:04.847450 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:04.847407 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:05.180512 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:05.180478 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:43:05.180958 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:05.180534 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:43:05.180958 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:05.180635 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 03:43:05.180958 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:05.180723 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls podName:210d8dd5-d21a-4349-92ee-2f55b6a0786c nodeName:}" failed. No retries permitted until 2026-07-09 03:43:13.180707066 +0000 UTC m=+63.812367145 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-55v58" (UID: "210d8dd5-d21a-4349-92ee-2f55b6a0786c") : secret "insights-runtime-extractor-tls" not found Jul 09 03:43:05.181102 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:05.181084 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/210d8dd5-d21a-4349-92ee-2f55b6a0786c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:43:05.248706 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:05.248666 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:05.443409 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:05.443306 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:06.046926 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:06.046856 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:06.050670 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:06.050635 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:43:06.050756 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:06.050699 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:43:06.051098 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:06.051084 2566 scope.go:117] "RemoveContainer" containerID="5313f92dc959b28f279cf18439af2720fb0610cca701441dc9fe3920ca9a7d44" Jul 09 03:43:06.051269 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:06.051250 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-gskrv_openshift-console-operator(d7993763-77e3-47b3-a32d-aa4b7c406413)\"" pod="openshift-console-operator/console-operator-68b757fb67-gskrv" podUID="d7993763-77e3-47b3-a32d-aa4b7c406413" Jul 09 03:43:06.244595 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:06.244554 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:06.447082 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:06.447042 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:06.495873 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:06.495816 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:06.648070 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:06.648034 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:06.848642 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:06.848550 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:06.948480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:06.948446 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:07.519142 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:07.519104 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:07.849884 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:07.849787 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:08.248034 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:08.248001 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:08.447249 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:08.447216 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:08.645386 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:08.645341 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:08.763386 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:08.763353 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:08.864456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:08.864419 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:10.028232 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:10.028183 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xpkvc\" (UID: \"c76314eb-b801-418f-9800-c7b0f42c0f97\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" Jul 09 03:43:10.028750 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:10.028331 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 03:43:10.028750 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:43:10.028400 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert podName:c76314eb-b801-418f-9800-c7b0f42c0f97 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:26.028384094 +0000 UTC m=+76.660044158 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xpkvc" (UID: "c76314eb-b801-418f-9800-c7b0f42c0f97") : secret "networking-console-plugin-cert" not found Jul 09 03:43:10.050494 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:10.050453 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:10.249684 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:10.249625 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:10.446710 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:10.446677 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:10.670431 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:10.670395 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:10.847840 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:10.847744 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:11.849815 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:11.849779 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:12.009613 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:12.009586 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s5kh8" Jul 09 03:43:12.247016 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:12.246977 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:12.449456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:12.449405 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:12.646576 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:12.646541 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:13.047054 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:13.046965 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:13.249384 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:13.249342 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:13.259522 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:13.259475 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:43:13.261869 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:13.261848 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/210d8dd5-d21a-4349-92ee-2f55b6a0786c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55v58\" (UID: \"210d8dd5-d21a-4349-92ee-2f55b6a0786c\") " pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:43:13.308165 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:13.308083 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-phtqr\"" Jul 09 03:43:13.316517 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:13.316487 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-55v58" Jul 09 03:43:13.449011 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:13.448977 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-55v58"] Jul 09 03:43:13.452621 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:43:13.452596 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod210d8dd5_d21a_4349_92ee_2f55b6a0786c.slice/crio-5665d3ece9faba278944185ce8dc11fc65c5e576d4200679af7642553957c167 WatchSource:0}: Error finding container 5665d3ece9faba278944185ce8dc11fc65c5e576d4200679af7642553957c167: Status 404 returned error can't find the container with id 5665d3ece9faba278944185ce8dc11fc65c5e576d4200679af7642553957c167 Jul 09 03:43:13.744391 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:13.744358 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:13.799196 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:13.799151 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:14.165626 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:14.165582 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-55v58" event={"ID":"210d8dd5-d21a-4349-92ee-2f55b6a0786c","Type":"ContainerStarted","Data":"5665d3ece9faba278944185ce8dc11fc65c5e576d4200679af7642553957c167"} Jul 09 03:43:15.685838 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:15.685788 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:43:15.685838 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:15.685850 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dd75\" (UniqueName: \"kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75\") pod \"network-check-target-vkw66\" (UID: \"ecce4016-016f-4060-8fcc-d350522156ad\") " pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:43:15.686411 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:15.685891 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs\") pod \"network-metrics-daemon-r9c5k\" (UID: \"c275241c-186f-492a-952a-cb6ed44c560f\") " pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:43:15.688577 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:15.688547 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 03:43:15.688950 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:15.688920 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dd75\" (UniqueName: \"kubernetes.io/projected/ecce4016-016f-4060-8fcc-d350522156ad-kube-api-access-4dd75\") pod \"network-check-target-vkw66\" (UID: \"ecce4016-016f-4060-8fcc-d350522156ad\") " pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:43:15.689622 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:15.689603 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 03:43:15.699630 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:15.699589 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69b8b6fa-1fda-4c62-81d1-c9b27231a274-original-pull-secret\") pod \"global-pull-secret-syncer-k9f2k\" (UID: \"69b8b6fa-1fda-4c62-81d1-c9b27231a274\") " pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:43:15.699630 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:15.699605 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c275241c-186f-492a-952a-cb6ed44c560f-metrics-certs\") pod \"network-metrics-daemon-r9c5k\" (UID: \"c275241c-186f-492a-952a-cb6ed44c560f\") " pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:43:15.974565 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:15.974482 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-54tq2\"" Jul 09 03:43:15.978938 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:15.978910 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9f2k" Jul 09 03:43:15.982042 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:15.982014 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r9c5k" Jul 09 03:43:15.986870 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:15.986845 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ch8mv\"" Jul 09 03:43:15.995398 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:15.995367 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:43:17.164442 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.164409 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r9c5k"] Jul 09 03:43:17.168159 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:43:17.168122 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc275241c_186f_492a_952a_cb6ed44c560f.slice/crio-4f78657b5673a1ecf786ef4d1e6c3ac4d67d411116f3fd2a903367541a6f04b5 WatchSource:0}: Error finding container 4f78657b5673a1ecf786ef4d1e6c3ac4d67d411116f3fd2a903367541a6f04b5: Status 404 returned error can't find the container with id 4f78657b5673a1ecf786ef4d1e6c3ac4d67d411116f3fd2a903367541a6f04b5 Jul 09 03:43:17.177365 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.177326 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r9c5k" event={"ID":"c275241c-186f-492a-952a-cb6ed44c560f","Type":"ContainerStarted","Data":"4f78657b5673a1ecf786ef4d1e6c3ac4d67d411116f3fd2a903367541a6f04b5"} Jul 09 03:43:17.178918 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.178887 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-55v58" event={"ID":"210d8dd5-d21a-4349-92ee-2f55b6a0786c","Type":"ContainerStarted","Data":"16869a4b98b3f83f483cc0d41e3e251e15f20c062951a8f9eb365a2c529de4e5"} Jul 09 03:43:17.183111 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.183082 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k9f2k"] Jul 09 03:43:17.186361 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:43:17.186329 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69b8b6fa_1fda_4c62_81d1_c9b27231a274.slice/crio-93a1450c4a3a24aabb886be216425551d1579d22f5ae945fd320e6484cccbdb9 WatchSource:0}: Error finding container 93a1450c4a3a24aabb886be216425551d1579d22f5ae945fd320e6484cccbdb9: Status 404 returned error can't find the container with id 93a1450c4a3a24aabb886be216425551d1579d22f5ae945fd320e6484cccbdb9 Jul 09 03:43:17.382338 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.382306 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vkw66"] Jul 09 03:43:17.384533 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:43:17.384503 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecce4016_016f_4060_8fcc_d350522156ad.slice/crio-ecfa289adbd3d77acae591ce0da03af8ae2854a89eb2edc4fc89833e244d6394 WatchSource:0}: Error finding container ecfa289adbd3d77acae591ce0da03af8ae2854a89eb2edc4fc89833e244d6394: Status 404 returned error can't find the container with id ecfa289adbd3d77acae591ce0da03af8ae2854a89eb2edc4fc89833e244d6394 Jul 09 03:43:17.605622 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.605584 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:43:17.605834 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.605635 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:43:17.605834 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.605733 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:43:17.605834 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.605765 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s29mn\" (UID: \"27fcabd6-d124-4abd-8b2d-86cc903a5693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" Jul 09 03:43:17.605834 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.605801 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:43:17.606440 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.606401 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93f8e14e-aea8-496e-b728-1554938d338a-service-ca-bundle\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:43:17.608265 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.608242 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93f8e14e-aea8-496e-b728-1554938d338a-metrics-certs\") pod \"router-default-7dfb6bc4d7-d7vz4\" (UID: \"93f8e14e-aea8-496e-b728-1554938d338a\") " pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:43:17.608387 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.608243 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d01b29b9-9680-4c59-acad-460f17b275f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tl5sq\" (UID: \"d01b29b9-9680-4c59-acad-460f17b275f4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:43:17.608387 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.608276 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27fcabd6-d124-4abd-8b2d-86cc903a5693-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s29mn\" (UID: \"27fcabd6-d124-4abd-8b2d-86cc903a5693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" Jul 09 03:43:17.608387 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.608340 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls\") pod \"image-registry-7f8c949fc8-jw5ks\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:43:17.706222 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.706176 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:43:17.708441 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.708408 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ca5948-6243-4377-9c91-a3da567aff5e-metrics-tls\") pod \"dns-default-mzcvn\" (UID: \"64ca5948-6243-4377-9c91-a3da567aff5e\") " pod="openshift-dns/dns-default-mzcvn" Jul 09 03:43:17.808607 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.808576 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nqczr\"" Jul 09 03:43:17.816845 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.816819 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:43:17.819193 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.819172 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4zwkd\"" Jul 09 03:43:17.827428 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.827401 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:43:17.842570 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.842545 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-qx2k9\"" Jul 09 03:43:17.850124 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.850099 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" Jul 09 03:43:17.859992 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.859969 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-bm4nv\"" Jul 09 03:43:17.868584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.868560 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" Jul 09 03:43:17.895842 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.895815 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hwlx6\"" Jul 09 03:43:17.903361 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:17.903330 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mzcvn" Jul 09 03:43:18.017022 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:43:18.016381 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88d0565d_8fda_4829_833e_43b2b5e36610.slice/crio-02945c9a15a691219e284f8b2d81d1a43c72e24f844dec237196c32cafcb3317 WatchSource:0}: Error finding container 02945c9a15a691219e284f8b2d81d1a43c72e24f844dec237196c32cafcb3317: Status 404 returned error can't find the container with id 02945c9a15a691219e284f8b2d81d1a43c72e24f844dec237196c32cafcb3317 Jul 09 03:43:18.019273 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.019105 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f8c949fc8-jw5ks"] Jul 09 03:43:18.041432 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.036699 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7dfb6bc4d7-d7vz4"] Jul 09 03:43:18.096947 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.085227 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq"] Jul 09 03:43:18.104370 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:43:18.104302 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd01b29b9_9680_4c59_acad_460f17b275f4.slice/crio-0d0c1c7f1aeccbf04a1c3c652f21f69ef46f410624978ed28e9350c37448bfa0 WatchSource:0}: Error finding container 0d0c1c7f1aeccbf04a1c3c652f21f69ef46f410624978ed28e9350c37448bfa0: Status 404 returned error can't find the container with id 0d0c1c7f1aeccbf04a1c3c652f21f69ef46f410624978ed28e9350c37448bfa0 Jul 09 03:43:18.140489 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.135851 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn"] Jul 09 03:43:18.180483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.180331 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mzcvn"] Jul 09 03:43:18.183520 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:43:18.183476 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64ca5948_6243_4377_9c91_a3da567aff5e.slice/crio-5eba383c9de0967f5d6b7ef186fce56286ae2c0242d6ea7070aab151bf97938d WatchSource:0}: Error finding container 5eba383c9de0967f5d6b7ef186fce56286ae2c0242d6ea7070aab151bf97938d: Status 404 returned error can't find the container with id 5eba383c9de0967f5d6b7ef186fce56286ae2c0242d6ea7070aab151bf97938d Jul 09 03:43:18.185182 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.185092 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k9f2k" event={"ID":"69b8b6fa-1fda-4c62-81d1-c9b27231a274","Type":"ContainerStarted","Data":"93a1450c4a3a24aabb886be216425551d1579d22f5ae945fd320e6484cccbdb9"} Jul 09 03:43:18.186614 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.186591 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" event={"ID":"93f8e14e-aea8-496e-b728-1554938d338a","Type":"ContainerStarted","Data":"8da0e66dad9e7aadd4243e9b29f3c80301f1cd5101cafb7bb9273c1c014c63d9"} Jul 09 03:43:18.188563 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.188534 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" event={"ID":"88d0565d-8fda-4829-833e-43b2b5e36610","Type":"ContainerStarted","Data":"02945c9a15a691219e284f8b2d81d1a43c72e24f844dec237196c32cafcb3317"} Jul 09 03:43:18.191600 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.191106 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vkw66" event={"ID":"ecce4016-016f-4060-8fcc-d350522156ad","Type":"ContainerStarted","Data":"370915724f7a2fbba6f33cc5950a830260c63a9fba9a4f3bbaf564c0a67bac4f"} Jul 09 03:43:18.191600 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.191148 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vkw66" event={"ID":"ecce4016-016f-4060-8fcc-d350522156ad","Type":"ContainerStarted","Data":"ecfa289adbd3d77acae591ce0da03af8ae2854a89eb2edc4fc89833e244d6394"} Jul 09 03:43:18.191600 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.191222 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:43:18.195292 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.195214 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-55v58" event={"ID":"210d8dd5-d21a-4349-92ee-2f55b6a0786c","Type":"ContainerStarted","Data":"8984cc0060abfad05ab92eb5bf2aaa4b958a3c81b513ceba749c43f4ab6adbc8"} Jul 09 03:43:18.196998 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.196929 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" event={"ID":"d01b29b9-9680-4c59-acad-460f17b275f4","Type":"ContainerStarted","Data":"0d0c1c7f1aeccbf04a1c3c652f21f69ef46f410624978ed28e9350c37448bfa0"} Jul 09 03:43:18.206696 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.206411 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vkw66" podStartSLOduration=67.206387157 podStartE2EDuration="1m7.206387157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:18.20605709 +0000 UTC m=+68.837717175" watchObservedRunningTime="2026-07-09 03:43:18.206387157 +0000 UTC m=+68.838047244" Jul 09 03:43:18.859394 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:18.858984 2566 scope.go:117] "RemoveContainer" containerID="5313f92dc959b28f279cf18439af2720fb0610cca701441dc9fe3920ca9a7d44" Jul 09 03:43:19.204833 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.204798 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 03:43:19.205308 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.204915 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-gskrv" event={"ID":"d7993763-77e3-47b3-a32d-aa4b7c406413","Type":"ContainerStarted","Data":"7381416f9b906a479a72129f39afc50ce8f88e2f6ba3a73db9aef60151485ca8"} Jul 09 03:43:19.205473 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.205424 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:43:19.210921 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.210884 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" event={"ID":"88d0565d-8fda-4829-833e-43b2b5e36610","Type":"ContainerStarted","Data":"9515b113bcbc29835c2448975b45243a8b2ba58724d121742b94776a609a83e2"} Jul 09 03:43:19.211090 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.210988 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:43:19.215478 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.213554 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" event={"ID":"27fcabd6-d124-4abd-8b2d-86cc903a5693","Type":"ContainerStarted","Data":"56dc7dbc8c4c6cda86a4551c14cbceb54e4ccd0c1fc87367193f190d5e3aa289"} Jul 09 03:43:19.216108 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.216064 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mzcvn" event={"ID":"64ca5948-6243-4377-9c91-a3da567aff5e","Type":"ContainerStarted","Data":"5eba383c9de0967f5d6b7ef186fce56286ae2c0242d6ea7070aab151bf97938d"} Jul 09 03:43:19.219233 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.219201 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" event={"ID":"93f8e14e-aea8-496e-b728-1554938d338a","Type":"ContainerStarted","Data":"071c1b04e55ea6a4796bb85fb25078beadc85d1ab96170ea15b3c02cde31ec2c"} Jul 09 03:43:19.225407 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.225242 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-gskrv" podStartSLOduration=49.418256398 podStartE2EDuration="57.225222434s" totalImagesPullingTime="7.806966036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:19.224501144 +0000 UTC m=+69.856161230" watchObservedRunningTime="2026-07-09 03:43:19.225222434 +0000 UTC m=+69.856882521" Jul 09 03:43:19.251777 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.251714 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" podStartSLOduration=68.251692025 podStartE2EDuration="1m8.251692025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:19.250667444 +0000 UTC m=+69.882327529" watchObservedRunningTime="2026-07-09 03:43:19.251692025 +0000 UTC m=+69.883352112" Jul 09 03:43:19.281344 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.280639 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" podStartSLOduration=59.28061327 podStartE2EDuration="59.28061327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:19.278895891 +0000 UTC m=+69.910555980" watchObservedRunningTime="2026-07-09 03:43:19.28061327 +0000 UTC m=+69.912273356" Jul 09 03:43:19.399366 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.398058 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696cdbd54-dfmlt"] Jul 09 03:43:19.400480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.400319 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696cdbd54-dfmlt" Jul 09 03:43:19.404638 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.404619 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 03:43:19.405339 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.404923 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 09 03:43:19.405339 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.404948 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw"] Jul 09 03:43:19.405339 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.404761 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 03:43:19.405906 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.404703 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5k925\"" Jul 09 03:43:19.407373 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.406917 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 03:43:19.407903 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.407683 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw"] Jul 09 03:43:19.410295 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.407842 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.411201 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.410634 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 09 03:43:19.411201 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.410992 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 09 03:43:19.411381 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.411264 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 09 03:43:19.414503 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.413726 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 09 03:43:19.416306 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.416287 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696cdbd54-dfmlt"] Jul 09 03:43:19.416404 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.416388 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" Jul 09 03:43:19.419906 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.419621 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 09 03:43:19.436517 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.432286 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw"] Jul 09 03:43:19.436517 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.432358 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw"] Jul 09 03:43:19.502929 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.502774 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:19.513964 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.513932 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-gskrv" Jul 09 03:43:19.525631 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.525591 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c20329a7-b348-4377-a0bb-48e21047dc1b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.525830 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.525688 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c20329a7-b348-4377-a0bb-48e21047dc1b-ca\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.525830 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.525718 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c20329a7-b348-4377-a0bb-48e21047dc1b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.525830 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.525774 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/80ea6010-1ff0-41b2-b294-0f7e37dd04d7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-696cdbd54-dfmlt\" (UID: \"80ea6010-1ff0-41b2-b294-0f7e37dd04d7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696cdbd54-dfmlt" Jul 09 03:43:19.525830 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.525801 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7htr\" (UniqueName: \"kubernetes.io/projected/c20329a7-b348-4377-a0bb-48e21047dc1b-kube-api-access-n7htr\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.526049 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.525849 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8291f753-f10e-4087-ad43-a4892e5113e4-tmp\") pod \"klusterlet-addon-workmgr-6847657df6-mpvbw\" (UID: \"8291f753-f10e-4087-ad43-a4892e5113e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" Jul 09 03:43:19.526049 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.525897 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8291f753-f10e-4087-ad43-a4892e5113e4-klusterlet-config\") pod \"klusterlet-addon-workmgr-6847657df6-mpvbw\" (UID: \"8291f753-f10e-4087-ad43-a4892e5113e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" Jul 09 03:43:19.526049 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.525995 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c20329a7-b348-4377-a0bb-48e21047dc1b-hub\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.526049 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.526024 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c20329a7-b348-4377-a0bb-48e21047dc1b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.526233 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.526050 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75lk9\" (UniqueName: \"kubernetes.io/projected/8291f753-f10e-4087-ad43-a4892e5113e4-kube-api-access-75lk9\") pod \"klusterlet-addon-workmgr-6847657df6-mpvbw\" (UID: \"8291f753-f10e-4087-ad43-a4892e5113e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" Jul 09 03:43:19.526233 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.526094 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp5vw\" (UniqueName: \"kubernetes.io/projected/80ea6010-1ff0-41b2-b294-0f7e37dd04d7-kube-api-access-jp5vw\") pod \"managed-serviceaccount-addon-agent-696cdbd54-dfmlt\" (UID: \"80ea6010-1ff0-41b2-b294-0f7e37dd04d7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696cdbd54-dfmlt" Jul 09 03:43:19.592153 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.591822 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:19.627481 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.626804 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c20329a7-b348-4377-a0bb-48e21047dc1b-hub\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.627481 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.626848 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c20329a7-b348-4377-a0bb-48e21047dc1b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.627481 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.626876 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75lk9\" (UniqueName: \"kubernetes.io/projected/8291f753-f10e-4087-ad43-a4892e5113e4-kube-api-access-75lk9\") pod \"klusterlet-addon-workmgr-6847657df6-mpvbw\" (UID: \"8291f753-f10e-4087-ad43-a4892e5113e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" Jul 09 03:43:19.627481 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.626912 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp5vw\" (UniqueName: \"kubernetes.io/projected/80ea6010-1ff0-41b2-b294-0f7e37dd04d7-kube-api-access-jp5vw\") pod \"managed-serviceaccount-addon-agent-696cdbd54-dfmlt\" (UID: \"80ea6010-1ff0-41b2-b294-0f7e37dd04d7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696cdbd54-dfmlt" Jul 09 03:43:19.627481 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.626971 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c20329a7-b348-4377-a0bb-48e21047dc1b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.627481 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.626999 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c20329a7-b348-4377-a0bb-48e21047dc1b-ca\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.627481 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.627023 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c20329a7-b348-4377-a0bb-48e21047dc1b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.627481 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.627070 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/80ea6010-1ff0-41b2-b294-0f7e37dd04d7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-696cdbd54-dfmlt\" (UID: \"80ea6010-1ff0-41b2-b294-0f7e37dd04d7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696cdbd54-dfmlt" Jul 09 03:43:19.627481 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.627096 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7htr\" (UniqueName: \"kubernetes.io/projected/c20329a7-b348-4377-a0bb-48e21047dc1b-kube-api-access-n7htr\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.627481 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.627141 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8291f753-f10e-4087-ad43-a4892e5113e4-tmp\") pod \"klusterlet-addon-workmgr-6847657df6-mpvbw\" (UID: \"8291f753-f10e-4087-ad43-a4892e5113e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" Jul 09 03:43:19.627481 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.627167 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8291f753-f10e-4087-ad43-a4892e5113e4-klusterlet-config\") pod \"klusterlet-addon-workmgr-6847657df6-mpvbw\" (UID: \"8291f753-f10e-4087-ad43-a4892e5113e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" Jul 09 03:43:19.629225 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.628276 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c20329a7-b348-4377-a0bb-48e21047dc1b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.629467 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.629425 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8291f753-f10e-4087-ad43-a4892e5113e4-tmp\") pod \"klusterlet-addon-workmgr-6847657df6-mpvbw\" (UID: \"8291f753-f10e-4087-ad43-a4892e5113e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" Jul 09 03:43:19.634371 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.633960 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c20329a7-b348-4377-a0bb-48e21047dc1b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.634371 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.634068 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c20329a7-b348-4377-a0bb-48e21047dc1b-hub\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.634604 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.634555 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c20329a7-b348-4377-a0bb-48e21047dc1b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.634800 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.634754 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c20329a7-b348-4377-a0bb-48e21047dc1b-ca\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.635136 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.635113 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/80ea6010-1ff0-41b2-b294-0f7e37dd04d7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-696cdbd54-dfmlt\" (UID: \"80ea6010-1ff0-41b2-b294-0f7e37dd04d7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696cdbd54-dfmlt" Jul 09 03:43:19.637327 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.636918 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8291f753-f10e-4087-ad43-a4892e5113e4-klusterlet-config\") pod \"klusterlet-addon-workmgr-6847657df6-mpvbw\" (UID: \"8291f753-f10e-4087-ad43-a4892e5113e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" Jul 09 03:43:19.637565 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.637520 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp5vw\" (UniqueName: \"kubernetes.io/projected/80ea6010-1ff0-41b2-b294-0f7e37dd04d7-kube-api-access-jp5vw\") pod \"managed-serviceaccount-addon-agent-696cdbd54-dfmlt\" (UID: \"80ea6010-1ff0-41b2-b294-0f7e37dd04d7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696cdbd54-dfmlt" Jul 09 03:43:19.642498 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.642451 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75lk9\" (UniqueName: \"kubernetes.io/projected/8291f753-f10e-4087-ad43-a4892e5113e4-kube-api-access-75lk9\") pod \"klusterlet-addon-workmgr-6847657df6-mpvbw\" (UID: \"8291f753-f10e-4087-ad43-a4892e5113e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" Jul 09 03:43:19.647264 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.647218 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7htr\" (UniqueName: \"kubernetes.io/projected/c20329a7-b348-4377-a0bb-48e21047dc1b-kube-api-access-n7htr\") pod \"cluster-proxy-proxy-agent-5cfff99b5-6vxcw\" (UID: \"c20329a7-b348-4377-a0bb-48e21047dc1b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.703413 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.703321 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:19.733093 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.731429 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-tkp5v"] Jul 09 03:43:19.735993 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.734992 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-tkp5v" Jul 09 03:43:19.739031 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.738589 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-tbz6h\"" Jul 09 03:43:19.739031 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.738696 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 03:43:19.739031 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.738872 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 03:43:19.748695 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.748602 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-tkp5v"] Jul 09 03:43:19.772932 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.772901 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696cdbd54-dfmlt" Jul 09 03:43:19.781158 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.780919 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" Jul 09 03:43:19.802032 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.801980 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:19.804701 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.804416 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" Jul 09 03:43:19.828162 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.828131 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:43:19.828604 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.828567 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc66q\" (UniqueName: \"kubernetes.io/projected/9fb07406-da35-4f28-a95c-558d2345c847-kube-api-access-gc66q\") pod \"downloads-685bfb5f64-tkp5v\" (UID: \"9fb07406-da35-4f28-a95c-558d2345c847\") " pod="openshift-console/downloads-685bfb5f64-tkp5v" Jul 09 03:43:19.832207 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.832161 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:43:19.902682 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.902626 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:19.929487 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.929375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc66q\" (UniqueName: \"kubernetes.io/projected/9fb07406-da35-4f28-a95c-558d2345c847-kube-api-access-gc66q\") pod \"downloads-685bfb5f64-tkp5v\" (UID: \"9fb07406-da35-4f28-a95c-558d2345c847\") " pod="openshift-console/downloads-685bfb5f64-tkp5v" Jul 09 03:43:19.938948 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:19.938913 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc66q\" (UniqueName: \"kubernetes.io/projected/9fb07406-da35-4f28-a95c-558d2345c847-kube-api-access-gc66q\") pod \"downloads-685bfb5f64-tkp5v\" (UID: \"9fb07406-da35-4f28-a95c-558d2345c847\") " pod="openshift-console/downloads-685bfb5f64-tkp5v" Jul 09 03:43:20.018164 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:20.017949 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:20.050966 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:20.050635 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-tkp5v" Jul 09 03:43:20.235938 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:20.235898 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r9c5k" event={"ID":"c275241c-186f-492a-952a-cb6ed44c560f","Type":"ContainerStarted","Data":"814e5b2db0d8ca73b301682ec130e882a3d4d8343da8fc9ec38121c5ad2fa443"} Jul 09 03:43:20.236342 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:20.235968 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r9c5k" event={"ID":"c275241c-186f-492a-952a-cb6ed44c560f","Type":"ContainerStarted","Data":"70e910fc84de6b032f0c54eb5bee9f56fe7a8375910407144994f9ec04fc92ca"} Jul 09 03:43:20.237149 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:20.236618 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:43:20.238415 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:20.238363 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7dfb6bc4d7-d7vz4" Jul 09 03:43:20.254441 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:20.253040 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-r9c5k" podStartSLOduration=68.167392268 podStartE2EDuration="1m10.253016975s" totalImagesPullingTime="2.085624707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:20.251817958 +0000 UTC m=+70.883478056" watchObservedRunningTime="2026-07-09 03:43:20.253016975 +0000 UTC m=+70.884677061" Jul 09 03:43:20.388485 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:20.388434 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:20.488045 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:20.487955 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:20.587426 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:20.587292 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:20.830261 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:20.830156 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:20.930882 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:20.930842 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:21.633038 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:21.633001 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:21.811031 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:21.810995 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:22.073802 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:22.073763 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:22.174541 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:22.174475 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:22.324659 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:22.324537 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:22.429126 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:22.429073 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:22.486749 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:22.486712 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:23.561064 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:23.561023 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:23.635226 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:23.635181 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:23.713152 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:23.713115 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:23.811399 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:23.811319 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:23.911396 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:23.911354 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:24.012073 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:24.012007 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:24.112455 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:24.112369 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:24.426332 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:24.425805 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:24.474007 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:24.473936 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-tkp5v"] Jul 09 03:43:24.489491 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:43:24.489457 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb07406_da35_4f28_a95c_558d2345c847.slice/crio-48d2cbb6d043d9557eeaeb0d8140ef5ea553ea06283c06e57dfd0fc22e262766 WatchSource:0}: Error finding container 48d2cbb6d043d9557eeaeb0d8140ef5ea553ea06283c06e57dfd0fc22e262766: Status 404 returned error can't find the container with id 48d2cbb6d043d9557eeaeb0d8140ef5ea553ea06283c06e57dfd0fc22e262766 Jul 09 03:43:24.528988 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:24.528699 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:24.546294 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:24.543462 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw"] Jul 09 03:43:24.553052 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:43:24.553012 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc20329a7_b348_4377_a0bb_48e21047dc1b.slice/crio-4369ff93bb5c35a33e1229a077aa22dc724d4d3e234a067ee57b7db62fb2f043 WatchSource:0}: Error finding container 4369ff93bb5c35a33e1229a077aa22dc724d4d3e234a067ee57b7db62fb2f043: Status 404 returned error can't find the container with id 4369ff93bb5c35a33e1229a077aa22dc724d4d3e234a067ee57b7db62fb2f043 Jul 09 03:43:24.574315 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:24.574290 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696cdbd54-dfmlt"] Jul 09 03:43:24.579526 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:43:24.579437 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80ea6010_1ff0_41b2_b294_0f7e37dd04d7.slice/crio-63ac7e65308d8e84b8a1fb542b29602a43e8cf04e13a8ae56914207f8ea4eea8 WatchSource:0}: Error finding container 63ac7e65308d8e84b8a1fb542b29602a43e8cf04e13a8ae56914207f8ea4eea8: Status 404 returned error can't find the container with id 63ac7e65308d8e84b8a1fb542b29602a43e8cf04e13a8ae56914207f8ea4eea8 Jul 09 03:43:24.593229 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:24.592411 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw"] Jul 09 03:43:24.819216 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:24.819176 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:24.918589 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:24.918555 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:25.004643 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.004608 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:25.127043 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.127004 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:25.227685 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.227572 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:25.258930 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.258898 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-55v58" event={"ID":"210d8dd5-d21a-4349-92ee-2f55b6a0786c","Type":"ContainerStarted","Data":"0e4f6d4cc03c7ceb9dc0169b1f0efa970bdf3cd76b076381fdcc5623f424b840"} Jul 09 03:43:25.260298 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.260268 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696cdbd54-dfmlt" event={"ID":"80ea6010-1ff0-41b2-b294-0f7e37dd04d7","Type":"ContainerStarted","Data":"63ac7e65308d8e84b8a1fb542b29602a43e8cf04e13a8ae56914207f8ea4eea8"} Jul 09 03:43:25.262377 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.262348 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" event={"ID":"27fcabd6-d124-4abd-8b2d-86cc903a5693","Type":"ContainerStarted","Data":"3f043a8c438254140fd5cd237603fd01ca7d6f65a1b28b9cbce13aea97401125"} Jul 09 03:43:25.262493 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.262402 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" event={"ID":"27fcabd6-d124-4abd-8b2d-86cc903a5693","Type":"ContainerStarted","Data":"94686e9ba615ad6cc6d8cec1fd48b340dbade0450fe18eced68a3fe090a65dbb"} Jul 09 03:43:25.264163 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.264140 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" event={"ID":"c20329a7-b348-4377-a0bb-48e21047dc1b","Type":"ContainerStarted","Data":"4369ff93bb5c35a33e1229a077aa22dc724d4d3e234a067ee57b7db62fb2f043"} Jul 09 03:43:25.265364 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.265337 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" event={"ID":"8291f753-f10e-4087-ad43-a4892e5113e4","Type":"ContainerStarted","Data":"e241b06591be863c13791a174555e7e0be426f3e46062e3f37e370b883c92234"} Jul 09 03:43:25.266575 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.266550 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-tkp5v" event={"ID":"9fb07406-da35-4f28-a95c-558d2345c847","Type":"ContainerStarted","Data":"48d2cbb6d043d9557eeaeb0d8140ef5ea553ea06283c06e57dfd0fc22e262766"} Jul 09 03:43:25.268177 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.268149 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" event={"ID":"d01b29b9-9680-4c59-acad-460f17b275f4","Type":"ContainerStarted","Data":"3af9fadb9b1f3b557da9478c091892899f923e953c1e264f83d9698a1162062f"} Jul 09 03:43:25.269791 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.269758 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k9f2k" event={"ID":"69b8b6fa-1fda-4c62-81d1-c9b27231a274","Type":"ContainerStarted","Data":"d259ca8c009197319ff0d7eaac7213ce44e1d1d4fb40ead033e8436c2cf8594a"} Jul 09 03:43:25.271738 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.271712 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mzcvn" event={"ID":"64ca5948-6243-4377-9c91-a3da567aff5e","Type":"ContainerStarted","Data":"308deb7944587900d1d3d10f776f5abc93496cd09740d974a376150af89f9c6b"} Jul 09 03:43:25.271848 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.271753 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mzcvn" event={"ID":"64ca5948-6243-4377-9c91-a3da567aff5e","Type":"ContainerStarted","Data":"1e0f874935d11d3cb9d52244f4b2c4a41a5936d24b26b63e6854828869361da6"} Jul 09 03:43:25.271904 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.271863 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mzcvn" Jul 09 03:43:25.279063 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.279004 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-55v58" podStartSLOduration=17.595982002 podStartE2EDuration="28.278982735s" totalImagesPullingTime="10.683000733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:42:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:25.276622191 +0000 UTC m=+75.908282278" watchObservedRunningTime="2026-07-09 03:43:25.278982735 +0000 UTC m=+75.910642820" Jul 09 03:43:25.284604 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.284574 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:25.292805 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.292747 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" podStartSLOduration=59.086993512 podStartE2EDuration="1m5.292726835s" totalImagesPullingTime="6.205733323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:25.291936048 +0000 UTC m=+75.923596135" watchObservedRunningTime="2026-07-09 03:43:25.292726835 +0000 UTC m=+75.924386923" Jul 09 03:43:25.332277 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.332212 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s29mn" podStartSLOduration=58.228344274 podStartE2EDuration="1m4.332189589s" totalImagesPullingTime="6.103845315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:25.314357663 +0000 UTC m=+75.946017751" watchObservedRunningTime="2026-07-09 03:43:25.332189589 +0000 UTC m=+75.963849677" Jul 09 03:43:25.333858 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.333811 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mzcvn" podStartSLOduration=34.20808582 podStartE2EDuration="40.333796131s" totalImagesPullingTime="6.125710311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:25.331856765 +0000 UTC m=+75.963516851" watchObservedRunningTime="2026-07-09 03:43:25.333796131 +0000 UTC m=+75.965456256" Jul 09 03:43:25.352805 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.352744 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k9f2k" podStartSLOduration=67.205876495 podStartE2EDuration="1m14.35272291s" totalImagesPullingTime="7.146846415s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:25.351595404 +0000 UTC m=+75.983255490" watchObservedRunningTime="2026-07-09 03:43:25.35272291 +0000 UTC m=+75.984382977" Jul 09 03:43:25.391111 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.391075 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:25.625212 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.625095 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:25.723066 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:25.723025 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:26.088205 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:26.088165 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xpkvc\" (UID: \"c76314eb-b801-418f-9800-c7b0f42c0f97\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" Jul 09 03:43:26.101303 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:26.101234 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c76314eb-b801-418f-9800-c7b0f42c0f97-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xpkvc\" (UID: \"c76314eb-b801-418f-9800-c7b0f42c0f97\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" Jul 09 03:43:26.218330 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:26.218111 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-72q9v\"" Jul 09 03:43:26.226513 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:26.226481 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" Jul 09 03:43:26.454675 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:26.453391 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc"] Jul 09 03:43:26.462055 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:43:26.461877 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc76314eb_b801_418f_9800_c7b0f42c0f97.slice/crio-33a76a33d6f2b4b92af7d55e1e6372ef378b4a3f3463a73fa6bd39c1a6945220 WatchSource:0}: Error finding container 33a76a33d6f2b4b92af7d55e1e6372ef378b4a3f3463a73fa6bd39c1a6945220: Status 404 returned error can't find the container with id 33a76a33d6f2b4b92af7d55e1e6372ef378b4a3f3463a73fa6bd39c1a6945220 Jul 09 03:43:27.294039 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:27.293941 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" event={"ID":"c76314eb-b801-418f-9800-c7b0f42c0f97","Type":"ContainerStarted","Data":"33a76a33d6f2b4b92af7d55e1e6372ef378b4a3f3463a73fa6bd39c1a6945220"} Jul 09 03:43:27.856166 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:27.856070 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:27.915533 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:27.915480 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:28.743492 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:28.743453 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:28.797188 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:28.797151 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:32.045510 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:32.045451 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:32.114198 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:32.114113 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:33.233422 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.233369 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:33.279946 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.279894 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:33.324578 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.324542 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" event={"ID":"c76314eb-b801-418f-9800-c7b0f42c0f97","Type":"ContainerStarted","Data":"7226212b9e93f898c321b85e04b2d162963d39836244aef94066187dc3f9b77f"} Jul 09 03:43:33.326503 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.326473 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696cdbd54-dfmlt" event={"ID":"80ea6010-1ff0-41b2-b294-0f7e37dd04d7","Type":"ContainerStarted","Data":"0aee8c1e73dcaf1d1befe02fc729efba01067990a129f40f8506846d159c067d"} Jul 09 03:43:33.329432 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.329342 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" event={"ID":"c20329a7-b348-4377-a0bb-48e21047dc1b","Type":"ContainerStarted","Data":"5558ce11480d74afc66e985634b1e30a7dec73107c49b5e6390b56d0995d4778"} Jul 09 03:43:33.329432 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.329383 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" event={"ID":"c20329a7-b348-4377-a0bb-48e21047dc1b","Type":"ContainerStarted","Data":"ccfde1c1a799c36f30c985d5cf75a80f3613b5579a98fa22cb860176cac2bb0b"} Jul 09 03:43:33.329432 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.329406 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" event={"ID":"c20329a7-b348-4377-a0bb-48e21047dc1b","Type":"ContainerStarted","Data":"85a5dc6f2f8eede60b70b162436a41e9d0e99642a3b8f3bdb20f700af316ee4f"} Jul 09 03:43:33.331218 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.331192 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" event={"ID":"8291f753-f10e-4087-ad43-a4892e5113e4","Type":"ContainerStarted","Data":"91f823d5fc3fcd1f696d701f5f03f173327425d69a8132f144302af173636894"} Jul 09 03:43:33.331369 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.331349 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" Jul 09 03:43:33.334841 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.334816 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" Jul 09 03:43:33.353790 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.353738 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696cdbd54-dfmlt" podStartSLOduration=6.5432183760000004 podStartE2EDuration="14.353720516s" totalImagesPullingTime="7.81050214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:33.353165644 +0000 UTC m=+83.984825725" watchObservedRunningTime="2026-07-09 03:43:33.353720516 +0000 UTC m=+83.985380602" Jul 09 03:43:33.354367 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.354338 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xpkvc" podStartSLOduration=33.424957216 podStartE2EDuration="39.354325074s" totalImagesPullingTime="5.929367858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:42:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:33.33959624 +0000 UTC m=+83.971256322" watchObservedRunningTime="2026-07-09 03:43:33.354325074 +0000 UTC m=+83.985985161" Jul 09 03:43:33.371803 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.371737 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cfff99b5-6vxcw" podStartSLOduration=6.505465562 podStartE2EDuration="14.371718153s" totalImagesPullingTime="7.866252591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:33.369956628 +0000 UTC m=+84.001616716" watchObservedRunningTime="2026-07-09 03:43:33.371718153 +0000 UTC m=+84.003378241" Jul 09 03:43:33.385477 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.385411 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6847657df6-mpvbw" podStartSLOduration=6.5689314450000005 podStartE2EDuration="14.38538629s" totalImagesPullingTime="7.816454845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:33.384549382 +0000 UTC m=+84.016209469" watchObservedRunningTime="2026-07-09 03:43:33.38538629 +0000 UTC m=+84.017046376" Jul 09 03:43:33.470054 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.470011 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:33.570672 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:33.570556 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:34.006944 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:34.006906 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:34.066527 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:34.066488 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:34.347631 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:34.347533 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:34.956021 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:34.955983 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:35.060631 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:35.060593 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:35.281045 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:35.280970 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mzcvn" Jul 09 03:43:35.912029 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:35.911976 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:36.012756 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:36.012714 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:36.063995 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:36.063930 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:37.042387 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:37.042356 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:37.143906 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:37.143868 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:37.822421 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:37.822374 2566 patch_prober.go:28] interesting pod/image-registry-7f8c949fc8-jw5ks container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 03:43:37.822585 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:37.822435 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" podUID="88d0565d-8fda-4829-833e-43b2b5e36610" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 03:43:38.464962 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:38.464931 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:40.242022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:40.241989 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:43:40.513956 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:40.513867 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:40.563117 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:40.563078 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:41.061243 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:41.061206 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:41.159472 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:41.159428 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:41.260463 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:41.260423 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:41.455166 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:41.455127 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:42.971472 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:42.971434 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:43.056490 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:43.056444 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:43.633864 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:43.633824 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:43.734867 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:43.734831 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:43.788563 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:43.788530 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" limit=1024 Jul 09 03:43:44.384022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:44.383992 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 03:43:44.384480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:44.384038 2566 generic.go:356] "Generic (PLEG): container finished" podID="d01b29b9-9680-4c59-acad-460f17b275f4" containerID="3af9fadb9b1f3b557da9478c091892899f923e953c1e264f83d9698a1162062f" exitCode=2 Jul 09 03:43:44.384480 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:44.384123 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" event={"ID":"d01b29b9-9680-4c59-acad-460f17b275f4","Type":"ContainerDied","Data":"3af9fadb9b1f3b557da9478c091892899f923e953c1e264f83d9698a1162062f"} Jul 09 03:43:44.384588 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:44.384514 2566 scope.go:117] "RemoveContainer" containerID="3af9fadb9b1f3b557da9478c091892899f923e953c1e264f83d9698a1162062f" Jul 09 03:43:45.391449 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:45.391409 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-tkp5v" event={"ID":"9fb07406-da35-4f28-a95c-558d2345c847","Type":"ContainerStarted","Data":"9b1d2a2f92c034150e3ccf3e4d55c029275bf825e19da84596313c7ee7ea2a28"} Jul 09 03:43:45.391957 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:45.391586 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-tkp5v" Jul 09 03:43:45.394065 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:45.394042 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 03:43:45.394205 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:45.394093 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tl5sq" event={"ID":"d01b29b9-9680-4c59-acad-460f17b275f4","Type":"ContainerStarted","Data":"f7c9ad73a483c37052fe83c67dc6aa041c1b35f9695ccd454d4d56bafb9e2e4d"} Jul 09 03:43:45.412578 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:45.412511 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-tkp5v" podStartSLOduration=6.201863367 podStartE2EDuration="26.412489792s" totalImagesPullingTime="20.210626425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:45.410036364 +0000 UTC m=+96.041696448" watchObservedRunningTime="2026-07-09 03:43:45.412489792 +0000 UTC m=+96.044149879" Jul 09 03:43:45.412841 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:45.412811 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-tkp5v" Jul 09 03:43:47.007163 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:47.007124 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f8c949fc8-jw5ks"] Jul 09 03:43:47.995160 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:47.993416 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ghz69"] Jul 09 03:43:48.028206 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.028171 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.031390 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.031362 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bhfm5\"" Jul 09 03:43:48.033013 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.032985 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 03:43:48.037665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.033309 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 03:43:48.037665 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.033799 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 03:43:48.043233 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.042773 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 03:43:48.088364 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.088318 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cb7fa39-6357-41a1-be15-fb41ca654e1c-sys\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.088552 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.088403 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-textfile\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.088552 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.088451 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.088552 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.088485 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbmdk\" (UniqueName: \"kubernetes.io/projected/5cb7fa39-6357-41a1-be15-fb41ca654e1c-kube-api-access-lbmdk\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.088552 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.088517 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-wtmp\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.088788 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.088559 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-tls\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.088788 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.088592 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-accelerators-collector-config\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.088788 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.088682 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5cb7fa39-6357-41a1-be15-fb41ca654e1c-root\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.088788 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.088735 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cb7fa39-6357-41a1-be15-fb41ca654e1c-metrics-client-ca\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.189494 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.189456 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-tls\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.189705 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.189512 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-accelerators-collector-config\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.189705 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.189560 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5cb7fa39-6357-41a1-be15-fb41ca654e1c-root\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.189705 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.189594 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cb7fa39-6357-41a1-be15-fb41ca654e1c-metrics-client-ca\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.189705 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.189624 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cb7fa39-6357-41a1-be15-fb41ca654e1c-sys\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.189705 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.189687 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-textfile\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.189973 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.189729 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.189973 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.189759 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbmdk\" (UniqueName: \"kubernetes.io/projected/5cb7fa39-6357-41a1-be15-fb41ca654e1c-kube-api-access-lbmdk\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.189973 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.189791 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-wtmp\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.190235 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.190007 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cb7fa39-6357-41a1-be15-fb41ca654e1c-sys\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.190235 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.190059 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5cb7fa39-6357-41a1-be15-fb41ca654e1c-root\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.190235 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.190194 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-accelerators-collector-config\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.190490 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.190470 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-textfile\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.190658 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.190625 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cb7fa39-6357-41a1-be15-fb41ca654e1c-metrics-client-ca\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.190782 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.190758 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-wtmp\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.192974 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.192952 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-tls\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.193124 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.193099 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cb7fa39-6357-41a1-be15-fb41ca654e1c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.202397 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.202367 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbmdk\" (UniqueName: \"kubernetes.io/projected/5cb7fa39-6357-41a1-be15-fb41ca654e1c-kube-api-access-lbmdk\") pod \"node-exporter-ghz69\" (UID: \"5cb7fa39-6357-41a1-be15-fb41ca654e1c\") " pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.348473 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.348386 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ghz69" Jul 09 03:43:48.381856 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:43:48.381817 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb7fa39_6357_41a1_be15_fb41ca654e1c.slice/crio-2f9509c58bcae6d4f6eeb6970a4486f430c0150320b2c6a70bf5aefc6422426c WatchSource:0}: Error finding container 2f9509c58bcae6d4f6eeb6970a4486f430c0150320b2c6a70bf5aefc6422426c: Status 404 returned error can't find the container with id 2f9509c58bcae6d4f6eeb6970a4486f430c0150320b2c6a70bf5aefc6422426c Jul 09 03:43:48.410663 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:48.410620 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ghz69" event={"ID":"5cb7fa39-6357-41a1-be15-fb41ca654e1c","Type":"ContainerStarted","Data":"2f9509c58bcae6d4f6eeb6970a4486f430c0150320b2c6a70bf5aefc6422426c"} Jul 09 03:43:49.223913 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:49.222864 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vkw66" Jul 09 03:43:50.422820 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:50.422776 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ghz69" event={"ID":"5cb7fa39-6357-41a1-be15-fb41ca654e1c","Type":"ContainerStarted","Data":"57dda51fd0a67e7332e683b17913e7260de4413f615ae2000b87e8c72393c8c5"} Jul 09 03:43:51.429699 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:51.429631 2566 generic.go:356] "Generic (PLEG): container finished" podID="5cb7fa39-6357-41a1-be15-fb41ca654e1c" containerID="57dda51fd0a67e7332e683b17913e7260de4413f615ae2000b87e8c72393c8c5" exitCode=0 Jul 09 03:43:51.430160 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:51.429736 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ghz69" event={"ID":"5cb7fa39-6357-41a1-be15-fb41ca654e1c","Type":"ContainerDied","Data":"57dda51fd0a67e7332e683b17913e7260de4413f615ae2000b87e8c72393c8c5"} Jul 09 03:43:52.437677 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:52.437614 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ghz69" event={"ID":"5cb7fa39-6357-41a1-be15-fb41ca654e1c","Type":"ContainerStarted","Data":"f6e6c2f18bf1655641e13679a582b48ae083e9eebe8c51b8f4504fba7a00b88f"} Jul 09 03:43:52.438151 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:52.437712 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ghz69" event={"ID":"5cb7fa39-6357-41a1-be15-fb41ca654e1c","Type":"ContainerStarted","Data":"521cf0526964603218662b79815ba611292b74abad5f877b9c3362e0409e8704"} Jul 09 03:43:52.456814 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:43:52.456731 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ghz69" podStartSLOduration=3.45335283 podStartE2EDuration="5.456713347s" totalImagesPullingTime="1.879162584s" totalInitContainerRuntime="124.197933ms" isStatefulPod=true podCreationTimestamp="2026-07-09 03:43:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:52.455370943 +0000 UTC m=+103.087031030" watchObservedRunningTime="2026-07-09 03:43:52.456713347 +0000 UTC m=+103.088373434" Jul 09 03:44:06.491533 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:06.491489 2566 generic.go:356] "Generic (PLEG): container finished" podID="a92ef34e-50ec-4d34-8d0f-a6e307d7a949" containerID="d8a28a5c9e9a3624e92afafdd9c75a5ae07bca549a383af57da13a14cea12095" exitCode=0 Jul 09 03:44:06.491970 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:06.491551 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" event={"ID":"a92ef34e-50ec-4d34-8d0f-a6e307d7a949","Type":"ContainerDied","Data":"d8a28a5c9e9a3624e92afafdd9c75a5ae07bca549a383af57da13a14cea12095"} Jul 09 03:44:06.491970 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:06.491892 2566 scope.go:117] "RemoveContainer" containerID="d8a28a5c9e9a3624e92afafdd9c75a5ae07bca549a383af57da13a14cea12095" Jul 09 03:44:07.498312 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:07.498271 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lbz6h" event={"ID":"a92ef34e-50ec-4d34-8d0f-a6e307d7a949","Type":"ContainerStarted","Data":"7cc3963e05c708e9d38043522204475d9fa9b62e60f601d0c687569abf4c4ace"} Jul 09 03:44:12.038678 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.038602 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" podUID="88d0565d-8fda-4829-833e-43b2b5e36610" containerName="registry" containerID="cri-o://9515b113bcbc29835c2448975b45243a8b2ba58724d121742b94776a609a83e2" gracePeriod=30 Jul 09 03:44:12.324859 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.324820 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:44:12.405902 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.405866 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88d0565d-8fda-4829-833e-43b2b5e36610-trusted-ca\") pod \"88d0565d-8fda-4829-833e-43b2b5e36610\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " Jul 09 03:44:12.406089 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.405919 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88d0565d-8fda-4829-833e-43b2b5e36610-registry-certificates\") pod \"88d0565d-8fda-4829-833e-43b2b5e36610\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " Jul 09 03:44:12.406089 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.405940 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls\") pod \"88d0565d-8fda-4829-833e-43b2b5e36610\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " Jul 09 03:44:12.406089 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.405963 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcmxb\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-kube-api-access-qcmxb\") pod \"88d0565d-8fda-4829-833e-43b2b5e36610\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " Jul 09 03:44:12.406089 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.406016 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88d0565d-8fda-4829-833e-43b2b5e36610-image-registry-private-configuration\") pod \"88d0565d-8fda-4829-833e-43b2b5e36610\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " Jul 09 03:44:12.406089 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.406071 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88d0565d-8fda-4829-833e-43b2b5e36610-ca-trust-extracted\") pod \"88d0565d-8fda-4829-833e-43b2b5e36610\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " Jul 09 03:44:12.406325 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.406095 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-bound-sa-token\") pod \"88d0565d-8fda-4829-833e-43b2b5e36610\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " Jul 09 03:44:12.406325 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.406131 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88d0565d-8fda-4829-833e-43b2b5e36610-installation-pull-secrets\") pod \"88d0565d-8fda-4829-833e-43b2b5e36610\" (UID: \"88d0565d-8fda-4829-833e-43b2b5e36610\") " Jul 09 03:44:12.407812 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.407376 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88d0565d-8fda-4829-833e-43b2b5e36610-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "88d0565d-8fda-4829-833e-43b2b5e36610" (UID: "88d0565d-8fda-4829-833e-43b2b5e36610"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 03:44:12.407812 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.407465 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88d0565d-8fda-4829-833e-43b2b5e36610-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "88d0565d-8fda-4829-833e-43b2b5e36610" (UID: "88d0565d-8fda-4829-833e-43b2b5e36610"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 03:44:12.407812 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.407749 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88d0565d-8fda-4829-833e-43b2b5e36610-trusted-ca\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:44:12.410771 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.410734 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88d0565d-8fda-4829-833e-43b2b5e36610-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "88d0565d-8fda-4829-833e-43b2b5e36610" (UID: "88d0565d-8fda-4829-833e-43b2b5e36610"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 03:44:12.412743 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.412192 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-kube-api-access-qcmxb" (OuterVolumeSpecName: "kube-api-access-qcmxb") pod "88d0565d-8fda-4829-833e-43b2b5e36610" (UID: "88d0565d-8fda-4829-833e-43b2b5e36610"). InnerVolumeSpecName "kube-api-access-qcmxb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 03:44:12.414715 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.412993 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "88d0565d-8fda-4829-833e-43b2b5e36610" (UID: "88d0565d-8fda-4829-833e-43b2b5e36610"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 03:44:12.414715 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.414358 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88d0565d-8fda-4829-833e-43b2b5e36610-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "88d0565d-8fda-4829-833e-43b2b5e36610" (UID: "88d0565d-8fda-4829-833e-43b2b5e36610"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 03:44:12.415948 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.415902 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "88d0565d-8fda-4829-833e-43b2b5e36610" (UID: "88d0565d-8fda-4829-833e-43b2b5e36610"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 03:44:12.420584 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.420556 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88d0565d-8fda-4829-833e-43b2b5e36610-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "88d0565d-8fda-4829-833e-43b2b5e36610" (UID: "88d0565d-8fda-4829-833e-43b2b5e36610"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 03:44:12.508704 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.508580 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88d0565d-8fda-4829-833e-43b2b5e36610-registry-certificates\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:44:12.508704 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.508615 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-registry-tls\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:44:12.508704 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.508625 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcmxb\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-kube-api-access-qcmxb\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:44:12.508704 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.508678 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88d0565d-8fda-4829-833e-43b2b5e36610-image-registry-private-configuration\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:44:12.508704 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.508693 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88d0565d-8fda-4829-833e-43b2b5e36610-ca-trust-extracted\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:44:12.508704 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.508706 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88d0565d-8fda-4829-833e-43b2b5e36610-bound-sa-token\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:44:12.509000 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.508716 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88d0565d-8fda-4829-833e-43b2b5e36610-installation-pull-secrets\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:44:12.520108 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.520069 2566 generic.go:356] "Generic (PLEG): container finished" podID="88d0565d-8fda-4829-833e-43b2b5e36610" containerID="9515b113bcbc29835c2448975b45243a8b2ba58724d121742b94776a609a83e2" exitCode=0 Jul 09 03:44:12.520279 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.520148 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" event={"ID":"88d0565d-8fda-4829-833e-43b2b5e36610","Type":"ContainerDied","Data":"9515b113bcbc29835c2448975b45243a8b2ba58724d121742b94776a609a83e2"} Jul 09 03:44:12.520279 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.520162 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" Jul 09 03:44:12.520279 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.520259 2566 scope.go:117] "RemoveContainer" containerID="9515b113bcbc29835c2448975b45243a8b2ba58724d121742b94776a609a83e2" Jul 09 03:44:12.520442 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.520248 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f8c949fc8-jw5ks" event={"ID":"88d0565d-8fda-4829-833e-43b2b5e36610","Type":"ContainerDied","Data":"02945c9a15a691219e284f8b2d81d1a43c72e24f844dec237196c32cafcb3317"} Jul 09 03:44:12.521920 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.521896 2566 generic.go:356] "Generic (PLEG): container finished" podID="f2c564d7-7702-4488-8faa-a7c27f1608c0" containerID="f346cae08cb0f5feee07bab178eb49ab8c103e07acc186eddbebeee6fd7f79cb" exitCode=0 Jul 09 03:44:12.522011 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.521939 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" event={"ID":"f2c564d7-7702-4488-8faa-a7c27f1608c0","Type":"ContainerDied","Data":"f346cae08cb0f5feee07bab178eb49ab8c103e07acc186eddbebeee6fd7f79cb"} Jul 09 03:44:12.522341 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.522324 2566 scope.go:117] "RemoveContainer" containerID="f346cae08cb0f5feee07bab178eb49ab8c103e07acc186eddbebeee6fd7f79cb" Jul 09 03:44:12.529746 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.529723 2566 scope.go:117] "RemoveContainer" containerID="9515b113bcbc29835c2448975b45243a8b2ba58724d121742b94776a609a83e2" Jul 09 03:44:12.530017 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:44:12.529998 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9515b113bcbc29835c2448975b45243a8b2ba58724d121742b94776a609a83e2\": container with ID starting with 9515b113bcbc29835c2448975b45243a8b2ba58724d121742b94776a609a83e2 not found: ID does not exist" containerID="9515b113bcbc29835c2448975b45243a8b2ba58724d121742b94776a609a83e2" Jul 09 03:44:12.530069 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.530026 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9515b113bcbc29835c2448975b45243a8b2ba58724d121742b94776a609a83e2"} err="failed to get container status \"9515b113bcbc29835c2448975b45243a8b2ba58724d121742b94776a609a83e2\": rpc error: code = NotFound desc = could not find container \"9515b113bcbc29835c2448975b45243a8b2ba58724d121742b94776a609a83e2\": container with ID starting with 9515b113bcbc29835c2448975b45243a8b2ba58724d121742b94776a609a83e2 not found: ID does not exist" Jul 09 03:44:12.555841 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.555806 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f8c949fc8-jw5ks"] Jul 09 03:44:12.561072 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:12.561045 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f8c949fc8-jw5ks"] Jul 09 03:44:13.528378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:13.528341 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tmxz8" event={"ID":"f2c564d7-7702-4488-8faa-a7c27f1608c0","Type":"ContainerStarted","Data":"07267a11ee13e71a56e42a1891f07a2d18c5e29ef55b936b37398cd857bceccc"} Jul 09 03:44:13.863594 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:13.863513 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88d0565d-8fda-4829-833e-43b2b5e36610" path="/var/lib/kubelet/pods/88d0565d-8fda-4829-833e-43b2b5e36610/volumes" Jul 09 03:44:16.546088 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:16.546050 2566 generic.go:356] "Generic (PLEG): container finished" podID="71b0a743-fa60-46e6-9fde-965caa5bdf86" containerID="32ed727179044cda0f0518951db50245952e7fd3582f325341debc671be742a2" exitCode=0 Jul 09 03:44:16.546446 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:16.546124 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" event={"ID":"71b0a743-fa60-46e6-9fde-965caa5bdf86","Type":"ContainerDied","Data":"32ed727179044cda0f0518951db50245952e7fd3582f325341debc671be742a2"} Jul 09 03:44:16.546515 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:16.546502 2566 scope.go:117] "RemoveContainer" containerID="32ed727179044cda0f0518951db50245952e7fd3582f325341debc671be742a2" Jul 09 03:44:17.552185 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:44:17.552153 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ddhcf" event={"ID":"71b0a743-fa60-46e6-9fde-965caa5bdf86","Type":"ContainerStarted","Data":"47b6216a9ddd98504058cfc2703677a6325af7b94dcb5e27daf4423ec9b12813"} Jul 09 03:47:09.792040 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:09.792013 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 03:47:09.796398 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:09.796375 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 03:47:09.796776 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:09.796753 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 03:47:09.801048 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:09.801030 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 03:47:09.807666 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:09.807629 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 03:47:37.052799 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.052759 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n"] Jul 09 03:47:37.054284 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.053218 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88d0565d-8fda-4829-833e-43b2b5e36610" containerName="registry" Jul 09 03:47:37.054284 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.053237 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="88d0565d-8fda-4829-833e-43b2b5e36610" containerName="registry" Jul 09 03:47:37.054284 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.053305 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="88d0565d-8fda-4829-833e-43b2b5e36610" containerName="registry" Jul 09 03:47:37.055179 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.055159 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.058043 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.058024 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 03:47:37.058188 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.058169 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 03:47:37.058830 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.058812 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gpbc5\"" Jul 09 03:47:37.058927 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.058858 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 03:47:37.058927 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.058921 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 03:47:37.069190 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.069164 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n"] Jul 09 03:47:37.114198 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.114158 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e15e6c75-3643-4caa-b658-6e7c30537173-manifests\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.114370 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.114261 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e15e6c75-3643-4caa-b658-6e7c30537173-apiservice-cert\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.114370 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.114301 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e15e6c75-3643-4caa-b658-6e7c30537173-webhook-cert\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.114370 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.114334 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e15e6c75-3643-4caa-b658-6e7c30537173-tmp\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.114370 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.114352 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjn2n\" (UniqueName: \"kubernetes.io/projected/e15e6c75-3643-4caa-b658-6e7c30537173-kube-api-access-vjn2n\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.214989 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.214944 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e15e6c75-3643-4caa-b658-6e7c30537173-manifests\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.215167 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.215007 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e15e6c75-3643-4caa-b658-6e7c30537173-apiservice-cert\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.215167 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.215038 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e15e6c75-3643-4caa-b658-6e7c30537173-webhook-cert\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.215167 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.215073 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e15e6c75-3643-4caa-b658-6e7c30537173-tmp\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.215167 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.215101 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjn2n\" (UniqueName: \"kubernetes.io/projected/e15e6c75-3643-4caa-b658-6e7c30537173-kube-api-access-vjn2n\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.215431 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.215409 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e15e6c75-3643-4caa-b658-6e7c30537173-manifests\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.217536 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.217499 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e15e6c75-3643-4caa-b658-6e7c30537173-tmp\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.217711 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.217550 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e15e6c75-3643-4caa-b658-6e7c30537173-webhook-cert\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.217711 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.217587 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e15e6c75-3643-4caa-b658-6e7c30537173-apiservice-cert\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.226276 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.226247 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjn2n\" (UniqueName: \"kubernetes.io/projected/e15e6c75-3643-4caa-b658-6e7c30537173-kube-api-access-vjn2n\") pod \"opendatahub-operator-controller-manager-645c4d5897-sgf4n\" (UID: \"e15e6c75-3643-4caa-b658-6e7c30537173\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.365392 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.365300 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:37.503626 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.503600 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n"] Jul 09 03:47:37.506359 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:47:37.506328 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode15e6c75_3643_4caa_b658_6e7c30537173.slice/crio-7f93661437badb6d0e2ff0b3322dadfc12c2ec026ce8d92a92e434d1f691c3e5 WatchSource:0}: Error finding container 7f93661437badb6d0e2ff0b3322dadfc12c2ec026ce8d92a92e434d1f691c3e5: Status 404 returned error can't find the container with id 7f93661437badb6d0e2ff0b3322dadfc12c2ec026ce8d92a92e434d1f691c3e5 Jul 09 03:47:37.508169 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:37.508146 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 03:47:38.343081 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:38.343038 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" event={"ID":"e15e6c75-3643-4caa-b658-6e7c30537173","Type":"ContainerStarted","Data":"7f93661437badb6d0e2ff0b3322dadfc12c2ec026ce8d92a92e434d1f691c3e5"} Jul 09 03:47:41.358996 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:41.358956 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" event={"ID":"e15e6c75-3643-4caa-b658-6e7c30537173","Type":"ContainerStarted","Data":"726e1bb050f9d56188c7884939f23d0d8ac6cbb28618ac7879fcac37af0afe5c"} Jul 09 03:47:42.365462 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:42.365426 2566 generic.go:356] "Generic (PLEG): container finished" podID="e15e6c75-3643-4caa-b658-6e7c30537173" containerID="726e1bb050f9d56188c7884939f23d0d8ac6cbb28618ac7879fcac37af0afe5c" exitCode=0 Jul 09 03:47:42.365906 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:42.365516 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" event={"ID":"e15e6c75-3643-4caa-b658-6e7c30537173","Type":"ContainerDied","Data":"726e1bb050f9d56188c7884939f23d0d8ac6cbb28618ac7879fcac37af0afe5c"} Jul 09 03:47:43.371568 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:43.371536 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" event={"ID":"e15e6c75-3643-4caa-b658-6e7c30537173","Type":"ContainerStarted","Data":"532188bd5aa3d04ff7705211438ab5eede0a16925e1514e95693a49fd3e7ea04"} Jul 09 03:47:43.371974 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:43.371627 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:47:43.394307 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:43.393763 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" podStartSLOduration=1.818598513 podStartE2EDuration="6.393739561s" totalImagesPullingTime="3.485067557s" totalInitContainerRuntime="1.090073491s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:47:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:47:43.391554025 +0000 UTC m=+334.023214108" watchObservedRunningTime="2026-07-09 03:47:43.393739561 +0000 UTC m=+334.025399649" Jul 09 03:47:54.378081 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:47:54.378026 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-sgf4n" Jul 09 03:48:01.639803 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.639764 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b"] Jul 09 03:48:01.649861 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.649828 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b" Jul 09 03:48:01.653875 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.653844 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 03:48:01.654028 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.653844 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 03:48:01.654028 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.653849 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-992kn\"" Jul 09 03:48:01.655424 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.655401 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b"] Jul 09 03:48:01.726429 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.726388 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-576x4\" (UniqueName: \"kubernetes.io/projected/ea8d949d-c7f5-43c5-8d70-7259a73dec30-kube-api-access-576x4\") pod \"kube-auth-proxy-5796d489b5-d9v5b\" (UID: \"ea8d949d-c7f5-43c5-8d70-7259a73dec30\") " pod="openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b" Jul 09 03:48:01.726618 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.726452 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea8d949d-c7f5-43c5-8d70-7259a73dec30-tls-certs\") pod \"kube-auth-proxy-5796d489b5-d9v5b\" (UID: \"ea8d949d-c7f5-43c5-8d70-7259a73dec30\") " pod="openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b" Jul 09 03:48:01.726618 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.726476 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea8d949d-c7f5-43c5-8d70-7259a73dec30-tmp\") pod \"kube-auth-proxy-5796d489b5-d9v5b\" (UID: \"ea8d949d-c7f5-43c5-8d70-7259a73dec30\") " pod="openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b" Jul 09 03:48:01.802475 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.802431 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-x2tlh"] Jul 09 03:48:01.805186 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.805159 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-x2tlh" Jul 09 03:48:01.807994 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.807965 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 09 03:48:01.808921 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.808903 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-wngsx\"" Jul 09 03:48:01.813948 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.813922 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-x2tlh"] Jul 09 03:48:01.827432 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.827388 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-576x4\" (UniqueName: \"kubernetes.io/projected/ea8d949d-c7f5-43c5-8d70-7259a73dec30-kube-api-access-576x4\") pod \"kube-auth-proxy-5796d489b5-d9v5b\" (UID: \"ea8d949d-c7f5-43c5-8d70-7259a73dec30\") " pod="openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b" Jul 09 03:48:01.827619 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.827472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea8d949d-c7f5-43c5-8d70-7259a73dec30-tls-certs\") pod \"kube-auth-proxy-5796d489b5-d9v5b\" (UID: \"ea8d949d-c7f5-43c5-8d70-7259a73dec30\") " pod="openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b" Jul 09 03:48:01.827619 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.827505 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea8d949d-c7f5-43c5-8d70-7259a73dec30-tmp\") pod \"kube-auth-proxy-5796d489b5-d9v5b\" (UID: \"ea8d949d-c7f5-43c5-8d70-7259a73dec30\") " pod="openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b" Jul 09 03:48:01.830406 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.830379 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea8d949d-c7f5-43c5-8d70-7259a73dec30-tmp\") pod \"kube-auth-proxy-5796d489b5-d9v5b\" (UID: \"ea8d949d-c7f5-43c5-8d70-7259a73dec30\") " pod="openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b" Jul 09 03:48:01.830725 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.830566 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea8d949d-c7f5-43c5-8d70-7259a73dec30-tls-certs\") pod \"kube-auth-proxy-5796d489b5-d9v5b\" (UID: \"ea8d949d-c7f5-43c5-8d70-7259a73dec30\") " pod="openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b" Jul 09 03:48:01.839866 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.839837 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-576x4\" (UniqueName: \"kubernetes.io/projected/ea8d949d-c7f5-43c5-8d70-7259a73dec30-kube-api-access-576x4\") pod \"kube-auth-proxy-5796d489b5-d9v5b\" (UID: \"ea8d949d-c7f5-43c5-8d70-7259a73dec30\") " pod="openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b" Jul 09 03:48:01.929008 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.928971 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b07779a1-f238-4a74-89d5-2af5a916fac8-cert\") pod \"odh-model-controller-7998c67dc4-x2tlh\" (UID: \"b07779a1-f238-4a74-89d5-2af5a916fac8\") " pod="opendatahub/odh-model-controller-7998c67dc4-x2tlh" Jul 09 03:48:01.929170 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.929105 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5jzt\" (UniqueName: \"kubernetes.io/projected/b07779a1-f238-4a74-89d5-2af5a916fac8-kube-api-access-z5jzt\") pod \"odh-model-controller-7998c67dc4-x2tlh\" (UID: \"b07779a1-f238-4a74-89d5-2af5a916fac8\") " pod="opendatahub/odh-model-controller-7998c67dc4-x2tlh" Jul 09 03:48:01.960515 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:01.960477 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b" Jul 09 03:48:02.030601 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.030493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5jzt\" (UniqueName: \"kubernetes.io/projected/b07779a1-f238-4a74-89d5-2af5a916fac8-kube-api-access-z5jzt\") pod \"odh-model-controller-7998c67dc4-x2tlh\" (UID: \"b07779a1-f238-4a74-89d5-2af5a916fac8\") " pod="opendatahub/odh-model-controller-7998c67dc4-x2tlh" Jul 09 03:48:02.030601 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.030547 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b07779a1-f238-4a74-89d5-2af5a916fac8-cert\") pod \"odh-model-controller-7998c67dc4-x2tlh\" (UID: \"b07779a1-f238-4a74-89d5-2af5a916fac8\") " pod="opendatahub/odh-model-controller-7998c67dc4-x2tlh" Jul 09 03:48:02.033427 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.033393 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b07779a1-f238-4a74-89d5-2af5a916fac8-cert\") pod \"odh-model-controller-7998c67dc4-x2tlh\" (UID: \"b07779a1-f238-4a74-89d5-2af5a916fac8\") " pod="opendatahub/odh-model-controller-7998c67dc4-x2tlh" Jul 09 03:48:02.040565 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.040533 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5jzt\" (UniqueName: \"kubernetes.io/projected/b07779a1-f238-4a74-89d5-2af5a916fac8-kube-api-access-z5jzt\") pod \"odh-model-controller-7998c67dc4-x2tlh\" (UID: \"b07779a1-f238-4a74-89d5-2af5a916fac8\") " pod="opendatahub/odh-model-controller-7998c67dc4-x2tlh" Jul 09 03:48:02.116586 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.116547 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-x2tlh" Jul 09 03:48:02.121393 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.121357 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b"] Jul 09 03:48:02.126573 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:48:02.126540 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8d949d_c7f5_43c5_8d70_7259a73dec30.slice/crio-232f29777efa74b8b30b55a5b7a99c39ae0cf8eeda64b11529fde705a416d792 WatchSource:0}: Error finding container 232f29777efa74b8b30b55a5b7a99c39ae0cf8eeda64b11529fde705a416d792: Status 404 returned error can't find the container with id 232f29777efa74b8b30b55a5b7a99c39ae0cf8eeda64b11529fde705a416d792 Jul 09 03:48:02.264299 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.264271 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-x2tlh"] Jul 09 03:48:02.266850 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:48:02.266819 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb07779a1_f238_4a74_89d5_2af5a916fac8.slice/crio-d4cc54afa44248f59f3c603452d56fd8907f6b8fbee7f5f3b0fb38eea5013696 WatchSource:0}: Error finding container d4cc54afa44248f59f3c603452d56fd8907f6b8fbee7f5f3b0fb38eea5013696: Status 404 returned error can't find the container with id d4cc54afa44248f59f3c603452d56fd8907f6b8fbee7f5f3b0fb38eea5013696 Jul 09 03:48:02.461873 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.461769 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-x2tlh" event={"ID":"b07779a1-f238-4a74-89d5-2af5a916fac8","Type":"ContainerStarted","Data":"d4cc54afa44248f59f3c603452d56fd8907f6b8fbee7f5f3b0fb38eea5013696"} Jul 09 03:48:02.462776 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.462747 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4trxg"] Jul 09 03:48:02.466013 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.465991 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b" event={"ID":"ea8d949d-c7f5-43c5-8d70-7259a73dec30","Type":"ContainerStarted","Data":"232f29777efa74b8b30b55a5b7a99c39ae0cf8eeda64b11529fde705a416d792"} Jul 09 03:48:02.466108 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.466075 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4trxg" Jul 09 03:48:02.468797 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.468776 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 09 03:48:02.476838 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.476812 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4trxg"] Jul 09 03:48:02.535333 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.535298 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ee4103b-2e52-4965-ada7-cbe517269855-tls-certs\") pod \"model-serving-api-54d89bd5fc-4trxg\" (UID: \"8ee4103b-2e52-4965-ada7-cbe517269855\") " pod="opendatahub/model-serving-api-54d89bd5fc-4trxg" Jul 09 03:48:02.535537 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.535349 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44psg\" (UniqueName: \"kubernetes.io/projected/8ee4103b-2e52-4965-ada7-cbe517269855-kube-api-access-44psg\") pod \"model-serving-api-54d89bd5fc-4trxg\" (UID: \"8ee4103b-2e52-4965-ada7-cbe517269855\") " pod="opendatahub/model-serving-api-54d89bd5fc-4trxg" Jul 09 03:48:02.636844 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.636810 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ee4103b-2e52-4965-ada7-cbe517269855-tls-certs\") pod \"model-serving-api-54d89bd5fc-4trxg\" (UID: \"8ee4103b-2e52-4965-ada7-cbe517269855\") " pod="opendatahub/model-serving-api-54d89bd5fc-4trxg" Jul 09 03:48:02.637016 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.636870 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44psg\" (UniqueName: \"kubernetes.io/projected/8ee4103b-2e52-4965-ada7-cbe517269855-kube-api-access-44psg\") pod \"model-serving-api-54d89bd5fc-4trxg\" (UID: \"8ee4103b-2e52-4965-ada7-cbe517269855\") " pod="opendatahub/model-serving-api-54d89bd5fc-4trxg" Jul 09 03:48:02.637070 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:48:02.637043 2566 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 09 03:48:02.637125 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:48:02.637114 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ee4103b-2e52-4965-ada7-cbe517269855-tls-certs podName:8ee4103b-2e52-4965-ada7-cbe517269855 nodeName:}" failed. No retries permitted until 2026-07-09 03:48:03.13709711 +0000 UTC m=+353.768757179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8ee4103b-2e52-4965-ada7-cbe517269855-tls-certs") pod "model-serving-api-54d89bd5fc-4trxg" (UID: "8ee4103b-2e52-4965-ada7-cbe517269855") : secret "model-serving-api-tls" not found Jul 09 03:48:02.647954 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.647924 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44psg\" (UniqueName: \"kubernetes.io/projected/8ee4103b-2e52-4965-ada7-cbe517269855-kube-api-access-44psg\") pod \"model-serving-api-54d89bd5fc-4trxg\" (UID: \"8ee4103b-2e52-4965-ada7-cbe517269855\") " pod="opendatahub/model-serving-api-54d89bd5fc-4trxg" Jul 09 03:48:02.696234 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.696191 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-79fzt"] Jul 09 03:48:02.699465 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.699443 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.703635 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.703616 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 03:48:02.703635 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.703622 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 03:48:02.703839 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.703822 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 03:48:02.704528 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.704507 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-n6b5h\"" Jul 09 03:48:02.706597 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.706351 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 03:48:02.727031 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.726951 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-79fzt"] Jul 09 03:48:02.839795 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.839752 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-cacerts\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.839980 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.839803 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.839980 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.839831 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-token\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.839980 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.839923 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.839980 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.839964 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6312faac-9a46-4e4b-8646-3f374c2fe8a6-local-certs\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.840196 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.840041 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.840196 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.840089 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdw8v\" (UniqueName: \"kubernetes.io/projected/6312faac-9a46-4e4b-8646-3f374c2fe8a6-kube-api-access-kdw8v\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.941691 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.941630 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-cacerts\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.941902 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.941702 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.941902 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.941729 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-token\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.941902 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.941793 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.941902 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.941820 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6312faac-9a46-4e4b-8646-3f374c2fe8a6-local-certs\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.941902 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.941875 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.942143 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.941924 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdw8v\" (UniqueName: \"kubernetes.io/projected/6312faac-9a46-4e4b-8646-3f374c2fe8a6-kube-api-access-kdw8v\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.942729 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.942614 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.945004 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.944973 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-cacerts\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.945329 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.945306 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.945785 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.945763 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.945890 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.945834 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6312faac-9a46-4e4b-8646-3f374c2fe8a6-local-certs\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.951128 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.951093 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdw8v\" (UniqueName: \"kubernetes.io/projected/6312faac-9a46-4e4b-8646-3f374c2fe8a6-kube-api-access-kdw8v\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:02.951633 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:02.951605 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-token\") pod \"istiod-openshift-gateway-7574459656-79fzt\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:03.009869 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:03.009548 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:03.145470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:03.145423 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ee4103b-2e52-4965-ada7-cbe517269855-tls-certs\") pod \"model-serving-api-54d89bd5fc-4trxg\" (UID: \"8ee4103b-2e52-4965-ada7-cbe517269855\") " pod="opendatahub/model-serving-api-54d89bd5fc-4trxg" Jul 09 03:48:03.149050 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:03.149016 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ee4103b-2e52-4965-ada7-cbe517269855-tls-certs\") pod \"model-serving-api-54d89bd5fc-4trxg\" (UID: \"8ee4103b-2e52-4965-ada7-cbe517269855\") " pod="opendatahub/model-serving-api-54d89bd5fc-4trxg" Jul 09 03:48:03.194108 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:03.194018 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-79fzt"] Jul 09 03:48:03.197484 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:48:03.197443 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6312faac_9a46_4e4b_8646_3f374c2fe8a6.slice/crio-ae1c54a4d4b053b82641a23deece58787209c719998df07c1ffad83b623cc05f WatchSource:0}: Error finding container ae1c54a4d4b053b82641a23deece58787209c719998df07c1ffad83b623cc05f: Status 404 returned error can't find the container with id ae1c54a4d4b053b82641a23deece58787209c719998df07c1ffad83b623cc05f Jul 09 03:48:03.377558 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:03.377462 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4trxg" Jul 09 03:48:03.475322 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:03.475237 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" event={"ID":"6312faac-9a46-4e4b-8646-3f374c2fe8a6","Type":"ContainerStarted","Data":"ae1c54a4d4b053b82641a23deece58787209c719998df07c1ffad83b623cc05f"} Jul 09 03:48:03.560593 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:03.560519 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4trxg"] Jul 09 03:48:03.563917 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:48:03.563885 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee4103b_2e52_4965_ada7_cbe517269855.slice/crio-8af0b95da81625cb47b185377667cd013d7d59528c90df4a8ddd662f8629c760 WatchSource:0}: Error finding container 8af0b95da81625cb47b185377667cd013d7d59528c90df4a8ddd662f8629c760: Status 404 returned error can't find the container with id 8af0b95da81625cb47b185377667cd013d7d59528c90df4a8ddd662f8629c760 Jul 09 03:48:04.482718 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:04.482681 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4trxg" event={"ID":"8ee4103b-2e52-4965-ada7-cbe517269855","Type":"ContainerStarted","Data":"8af0b95da81625cb47b185377667cd013d7d59528c90df4a8ddd662f8629c760"} Jul 09 03:48:07.550884 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:07.550841 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 03:48:07.551246 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:07.550909 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 03:48:08.505600 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.505559 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-x2tlh" event={"ID":"b07779a1-f238-4a74-89d5-2af5a916fac8","Type":"ContainerStarted","Data":"a94a1113b38a1be56d4ec3f501a824af1a08e63c0f7dcc8fbfdf161372216a55"} Jul 09 03:48:08.505907 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.505634 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-x2tlh" Jul 09 03:48:08.507252 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.507227 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" event={"ID":"6312faac-9a46-4e4b-8646-3f374c2fe8a6","Type":"ContainerStarted","Data":"6000f17168556703e2dcc2ec1ee9bf320da286acd8dcf0bd9b5640618d67db41"} Jul 09 03:48:08.507371 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.507343 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:08.509436 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.509393 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4trxg" event={"ID":"8ee4103b-2e52-4965-ada7-cbe517269855","Type":"ContainerStarted","Data":"c6515c6d08066c0307b621496f685bc4b7d6b22cefb7dc2492f9290e35658a06"} Jul 09 03:48:08.509574 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.509548 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-4trxg" Jul 09 03:48:08.509635 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.509603 2566 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-79fzt container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 03:48:08.509785 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.509641 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" podUID="6312faac-9a46-4e4b-8646-3f374c2fe8a6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 03:48:08.511260 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.511235 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b" event={"ID":"ea8d949d-c7f5-43c5-8d70-7259a73dec30","Type":"ContainerStarted","Data":"6de60cf26af651eeae03d8bb50a1e4a0d7ff4c88379e1edfdfe7ee729a463d46"} Jul 09 03:48:08.525563 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.525506 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-x2tlh" podStartSLOduration=2.252366814 podStartE2EDuration="7.525490701s" totalImagesPullingTime="5.273123887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:48:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:48:08.52385891 +0000 UTC m=+359.155518996" watchObservedRunningTime="2026-07-09 03:48:08.525490701 +0000 UTC m=+359.157150786" Jul 09 03:48:08.564413 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.564352 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-4trxg" podStartSLOduration=2.589567492 podStartE2EDuration="6.564336105s" totalImagesPullingTime="3.974768613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:48:08.562699289 +0000 UTC m=+359.194359376" watchObservedRunningTime="2026-07-09 03:48:08.564336105 +0000 UTC m=+359.195996251" Jul 09 03:48:08.598234 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.598168 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5796d489b5-d9v5b" podStartSLOduration=3.148435252 podStartE2EDuration="7.598150177s" totalImagesPullingTime="4.449714925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:48:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:48:08.597935997 +0000 UTC m=+359.229596084" watchObservedRunningTime="2026-07-09 03:48:08.598150177 +0000 UTC m=+359.229810265" Jul 09 03:48:08.646403 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.646345 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" podStartSLOduration=2.29609765 podStartE2EDuration="6.64632686s" totalImagesPullingTime="4.35022921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:48:08.644417365 +0000 UTC m=+359.276077452" watchObservedRunningTime="2026-07-09 03:48:08.64632686 +0000 UTC m=+359.277986945" Jul 09 03:48:08.882059 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.881969 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686"] Jul 09 03:48:08.885767 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.885730 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:08.888436 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.888405 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-lsqkt\"" Jul 09 03:48:08.898261 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:08.898234 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686"] Jul 09 03:48:09.000668 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.000604 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.000668 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.000667 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.000910 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.000706 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbdvn\" (UniqueName: \"kubernetes.io/projected/5070aff3-9714-49c1-abb4-b5b320d8f0b7-kube-api-access-sbdvn\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.000910 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.000789 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.000910 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.000872 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.001063 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.000925 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.001063 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.000943 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.001063 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.000977 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.001063 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.001002 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.102356 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.102310 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.102526 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.102363 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.102526 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.102404 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbdvn\" (UniqueName: \"kubernetes.io/projected/5070aff3-9714-49c1-abb4-b5b320d8f0b7-kube-api-access-sbdvn\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.102526 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.102444 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.102706 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.102563 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.102706 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.102630 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.102706 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.102673 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.102844 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.102731 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.102844 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.102772 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.102844 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.102812 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.102844 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.102837 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.103143 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.103117 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.103281 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.103226 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.103581 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.103551 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.104997 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.104970 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.105283 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.105262 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.120798 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.120742 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbdvn\" (UniqueName: \"kubernetes.io/projected/5070aff3-9714-49c1-abb4-b5b320d8f0b7-kube-api-access-sbdvn\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.137359 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.137286 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5070aff3-9714-49c1-abb4-b5b320d8f0b7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678fm686\" (UID: \"5070aff3-9714-49c1-abb4-b5b320d8f0b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.197312 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.197274 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:09.363231 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.363200 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686"] Jul 09 03:48:09.365791 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:48:09.365758 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5070aff3_9714_49c1_abb4_b5b320d8f0b7.slice/crio-0581b16d1676feac75d63ca1951bdaba04a98f9954570550d4a970abb8441ff1 WatchSource:0}: Error finding container 0581b16d1676feac75d63ca1951bdaba04a98f9954570550d4a970abb8441ff1: Status 404 returned error can't find the container with id 0581b16d1676feac75d63ca1951bdaba04a98f9954570550d4a970abb8441ff1 Jul 09 03:48:09.516679 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.516632 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" event={"ID":"5070aff3-9714-49c1-abb4-b5b320d8f0b7","Type":"ContainerStarted","Data":"0581b16d1676feac75d63ca1951bdaba04a98f9954570550d4a970abb8441ff1"} Jul 09 03:48:09.517695 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:09.517677 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:11.554530 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:11.554497 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd"] Jul 09 03:48:11.558316 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:11.558292 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd" Jul 09 03:48:11.561569 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:11.561548 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 09 03:48:11.561698 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:11.561554 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-j7dc8\"" Jul 09 03:48:11.570749 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:11.570695 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd"] Jul 09 03:48:11.734212 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:11.734166 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a00fb81-09f2-4b20-b6e4-cea0d989aaf1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lljdd\" (UID: \"2a00fb81-09f2-4b20-b6e4-cea0d989aaf1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd" Jul 09 03:48:11.734386 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:11.734217 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkjnp\" (UniqueName: \"kubernetes.io/projected/2a00fb81-09f2-4b20-b6e4-cea0d989aaf1-kube-api-access-tkjnp\") pod \"llmisvc-controller-manager-777c6bd7d5-lljdd\" (UID: \"2a00fb81-09f2-4b20-b6e4-cea0d989aaf1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd" Jul 09 03:48:11.835224 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:11.835123 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a00fb81-09f2-4b20-b6e4-cea0d989aaf1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lljdd\" (UID: \"2a00fb81-09f2-4b20-b6e4-cea0d989aaf1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd" Jul 09 03:48:11.835224 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:11.835185 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkjnp\" (UniqueName: \"kubernetes.io/projected/2a00fb81-09f2-4b20-b6e4-cea0d989aaf1-kube-api-access-tkjnp\") pod \"llmisvc-controller-manager-777c6bd7d5-lljdd\" (UID: \"2a00fb81-09f2-4b20-b6e4-cea0d989aaf1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd" Jul 09 03:48:11.835457 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:48:11.835306 2566 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 09 03:48:11.835457 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:48:11.835386 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a00fb81-09f2-4b20-b6e4-cea0d989aaf1-cert podName:2a00fb81-09f2-4b20-b6e4-cea0d989aaf1 nodeName:}" failed. No retries permitted until 2026-07-09 03:48:12.335363879 +0000 UTC m=+362.967023965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2a00fb81-09f2-4b20-b6e4-cea0d989aaf1-cert") pod "llmisvc-controller-manager-777c6bd7d5-lljdd" (UID: "2a00fb81-09f2-4b20-b6e4-cea0d989aaf1") : secret "llmisvc-webhook-server-cert" not found Jul 09 03:48:11.853276 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:11.853240 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkjnp\" (UniqueName: \"kubernetes.io/projected/2a00fb81-09f2-4b20-b6e4-cea0d989aaf1-kube-api-access-tkjnp\") pod \"llmisvc-controller-manager-777c6bd7d5-lljdd\" (UID: \"2a00fb81-09f2-4b20-b6e4-cea0d989aaf1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd" Jul 09 03:48:12.339552 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:12.339502 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a00fb81-09f2-4b20-b6e4-cea0d989aaf1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lljdd\" (UID: \"2a00fb81-09f2-4b20-b6e4-cea0d989aaf1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd" Jul 09 03:48:12.342586 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:12.342547 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a00fb81-09f2-4b20-b6e4-cea0d989aaf1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lljdd\" (UID: \"2a00fb81-09f2-4b20-b6e4-cea0d989aaf1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd" Jul 09 03:48:12.468514 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:12.468465 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd" Jul 09 03:48:12.603134 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:12.603063 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd"] Jul 09 03:48:12.605181 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:48:12.605149 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2a00fb81_09f2_4b20_b6e4_cea0d989aaf1.slice/crio-b6e4033707891aca04af295490081da547463b831bd117511fc777237f26c58e WatchSource:0}: Error finding container b6e4033707891aca04af295490081da547463b831bd117511fc777237f26c58e: Status 404 returned error can't find the container with id b6e4033707891aca04af295490081da547463b831bd117511fc777237f26c58e Jul 09 03:48:13.538670 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:13.538620 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd" event={"ID":"2a00fb81-09f2-4b20-b6e4-cea0d989aaf1","Type":"ContainerStarted","Data":"b6e4033707891aca04af295490081da547463b831bd117511fc777237f26c58e"} Jul 09 03:48:14.018814 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.018774 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-79fzt"] Jul 09 03:48:14.019299 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.018999 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" podUID="6312faac-9a46-4e4b-8646-3f374c2fe8a6" containerName="discovery" containerID="cri-o://6000f17168556703e2dcc2ec1ee9bf320da286acd8dcf0bd9b5640618d67db41" gracePeriod=30 Jul 09 03:48:14.293754 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.293723 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:14.357621 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.357577 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-csr-dns-cert\") pod \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " Jul 09 03:48:14.357847 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.357632 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdw8v\" (UniqueName: \"kubernetes.io/projected/6312faac-9a46-4e4b-8646-3f374c2fe8a6-kube-api-access-kdw8v\") pod \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " Jul 09 03:48:14.357847 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.357691 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-cacerts\") pod \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " Jul 09 03:48:14.357847 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.357726 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-kubeconfig\") pod \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " Jul 09 03:48:14.357847 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.357750 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-token\") pod \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " Jul 09 03:48:14.357847 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.357772 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6312faac-9a46-4e4b-8646-3f374c2fe8a6-local-certs\") pod \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " Jul 09 03:48:14.357847 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.357804 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-csr-ca-configmap\") pod \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\" (UID: \"6312faac-9a46-4e4b-8646-3f374c2fe8a6\") " Jul 09 03:48:14.358384 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.358351 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "6312faac-9a46-4e4b-8646-3f374c2fe8a6" (UID: "6312faac-9a46-4e4b-8646-3f374c2fe8a6"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 03:48:14.360754 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.360719 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-cacerts" (OuterVolumeSpecName: "cacerts") pod "6312faac-9a46-4e4b-8646-3f374c2fe8a6" (UID: "6312faac-9a46-4e4b-8646-3f374c2fe8a6"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 03:48:14.360850 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.360719 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6312faac-9a46-4e4b-8646-3f374c2fe8a6-local-certs" (OuterVolumeSpecName: "local-certs") pod "6312faac-9a46-4e4b-8646-3f374c2fe8a6" (UID: "6312faac-9a46-4e4b-8646-3f374c2fe8a6"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 03:48:14.360850 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.360800 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "6312faac-9a46-4e4b-8646-3f374c2fe8a6" (UID: "6312faac-9a46-4e4b-8646-3f374c2fe8a6"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 03:48:14.360850 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.360813 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "6312faac-9a46-4e4b-8646-3f374c2fe8a6" (UID: "6312faac-9a46-4e4b-8646-3f374c2fe8a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 03:48:14.360850 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.360822 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "6312faac-9a46-4e4b-8646-3f374c2fe8a6" (UID: "6312faac-9a46-4e4b-8646-3f374c2fe8a6"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 03:48:14.360850 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.360825 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6312faac-9a46-4e4b-8646-3f374c2fe8a6-kube-api-access-kdw8v" (OuterVolumeSpecName: "kube-api-access-kdw8v") pod "6312faac-9a46-4e4b-8646-3f374c2fe8a6" (UID: "6312faac-9a46-4e4b-8646-3f374c2fe8a6"). InnerVolumeSpecName "kube-api-access-kdw8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 03:48:14.458687 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.458626 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdw8v\" (UniqueName: \"kubernetes.io/projected/6312faac-9a46-4e4b-8646-3f374c2fe8a6-kube-api-access-kdw8v\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:48:14.458687 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.458683 2566 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-cacerts\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:48:14.458687 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.458695 2566 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-kubeconfig\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:48:14.458914 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.458704 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-token\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:48:14.458914 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.458714 2566 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6312faac-9a46-4e4b-8646-3f374c2fe8a6-local-certs\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:48:14.458914 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.458722 2566 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-csr-ca-configmap\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:48:14.458914 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.458733 2566 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6312faac-9a46-4e4b-8646-3f374c2fe8a6-istio-csr-dns-cert\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:48:14.546132 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.546042 2566 generic.go:356] "Generic (PLEG): container finished" podID="6312faac-9a46-4e4b-8646-3f374c2fe8a6" containerID="6000f17168556703e2dcc2ec1ee9bf320da286acd8dcf0bd9b5640618d67db41" exitCode=0 Jul 09 03:48:14.546132 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.546102 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" Jul 09 03:48:14.546348 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.546145 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" event={"ID":"6312faac-9a46-4e4b-8646-3f374c2fe8a6","Type":"ContainerDied","Data":"6000f17168556703e2dcc2ec1ee9bf320da286acd8dcf0bd9b5640618d67db41"} Jul 09 03:48:14.546348 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.546250 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-79fzt" event={"ID":"6312faac-9a46-4e4b-8646-3f374c2fe8a6","Type":"ContainerDied","Data":"ae1c54a4d4b053b82641a23deece58787209c719998df07c1ffad83b623cc05f"} Jul 09 03:48:14.546348 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.546283 2566 scope.go:117] "RemoveContainer" containerID="6000f17168556703e2dcc2ec1ee9bf320da286acd8dcf0bd9b5640618d67db41" Jul 09 03:48:14.556352 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.556336 2566 scope.go:117] "RemoveContainer" containerID="6000f17168556703e2dcc2ec1ee9bf320da286acd8dcf0bd9b5640618d67db41" Jul 09 03:48:14.556617 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:48:14.556599 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6000f17168556703e2dcc2ec1ee9bf320da286acd8dcf0bd9b5640618d67db41\": container with ID starting with 6000f17168556703e2dcc2ec1ee9bf320da286acd8dcf0bd9b5640618d67db41 not found: ID does not exist" containerID="6000f17168556703e2dcc2ec1ee9bf320da286acd8dcf0bd9b5640618d67db41" Jul 09 03:48:14.556690 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.556626 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6000f17168556703e2dcc2ec1ee9bf320da286acd8dcf0bd9b5640618d67db41"} err="failed to get container status \"6000f17168556703e2dcc2ec1ee9bf320da286acd8dcf0bd9b5640618d67db41\": rpc error: code = NotFound desc = could not find container \"6000f17168556703e2dcc2ec1ee9bf320da286acd8dcf0bd9b5640618d67db41\": container with ID starting with 6000f17168556703e2dcc2ec1ee9bf320da286acd8dcf0bd9b5640618d67db41 not found: ID does not exist" Jul 09 03:48:14.570178 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.570151 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-79fzt"] Jul 09 03:48:14.574159 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:14.574133 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-79fzt"] Jul 09 03:48:15.865413 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:15.865382 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6312faac-9a46-4e4b-8646-3f374c2fe8a6" path="/var/lib/kubelet/pods/6312faac-9a46-4e4b-8646-3f374c2fe8a6/volumes" Jul 09 03:48:16.428834 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:16.428788 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 03:48:16.428961 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:16.428871 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 03:48:16.428961 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:16.428912 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 03:48:16.559807 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:16.559767 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" event={"ID":"5070aff3-9714-49c1-abb4-b5b320d8f0b7","Type":"ContainerStarted","Data":"a409f462502010f6d77e1e28b4115464e2add852cd8c395f7d41f1ba8acd8a04"} Jul 09 03:48:16.585291 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:16.585238 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" podStartSLOduration=1.524686402 podStartE2EDuration="8.585217739s" totalImagesPullingTime="7.060531337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:48:16.581715074 +0000 UTC m=+367.213375158" watchObservedRunningTime="2026-07-09 03:48:16.585217739 +0000 UTC m=+367.216877826" Jul 09 03:48:17.198051 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:17.198008 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:17.202601 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:17.202569 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:17.565757 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:17.565662 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd" event={"ID":"2a00fb81-09f2-4b20-b6e4-cea0d989aaf1","Type":"ContainerStarted","Data":"98ff357cddf9af78158e98328b006cf9553b8e080dc8217ac20049cb987ffb3f"} Jul 09 03:48:17.566037 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:17.566017 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:17.566108 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:17.566071 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd" Jul 09 03:48:17.566856 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:17.566838 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678fm686" Jul 09 03:48:17.584965 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:17.584910 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd" podStartSLOduration=2.760669839 podStartE2EDuration="6.584895016s" totalImagesPullingTime="3.824225177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:48:17.58328319 +0000 UTC m=+368.214943277" watchObservedRunningTime="2026-07-09 03:48:17.584895016 +0000 UTC m=+368.216555101" Jul 09 03:48:19.519845 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:19.519812 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-x2tlh" Jul 09 03:48:19.522316 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:19.522296 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-4trxg" Jul 09 03:48:49.581282 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:49.581250 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lljdd" Jul 09 03:48:57.442958 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.442924 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sckj8"] Jul 09 03:48:57.443386 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.443293 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6312faac-9a46-4e4b-8646-3f374c2fe8a6" containerName="discovery" Jul 09 03:48:57.443386 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.443303 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6312faac-9a46-4e4b-8646-3f374c2fe8a6" containerName="discovery" Jul 09 03:48:57.443386 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.443354 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6312faac-9a46-4e4b-8646-3f374c2fe8a6" containerName="discovery" Jul 09 03:48:57.453854 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.453825 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sckj8" Jul 09 03:48:57.459022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.458996 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 03:48:57.459134 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.459111 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-sg9bn\"" Jul 09 03:48:57.459447 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.459429 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 03:48:57.474790 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.474758 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sckj8"] Jul 09 03:48:57.634131 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.634085 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp5bx\" (UniqueName: \"kubernetes.io/projected/e3c18953-5efb-4c9b-9e8a-b954fd74863a-kube-api-access-dp5bx\") pod \"kuadrant-operator-catalog-sckj8\" (UID: \"e3c18953-5efb-4c9b-9e8a-b954fd74863a\") " pod="kuadrant-system/kuadrant-operator-catalog-sckj8" Jul 09 03:48:57.735635 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.735532 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp5bx\" (UniqueName: \"kubernetes.io/projected/e3c18953-5efb-4c9b-9e8a-b954fd74863a-kube-api-access-dp5bx\") pod \"kuadrant-operator-catalog-sckj8\" (UID: \"e3c18953-5efb-4c9b-9e8a-b954fd74863a\") " pod="kuadrant-system/kuadrant-operator-catalog-sckj8" Jul 09 03:48:57.743566 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.743530 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp5bx\" (UniqueName: \"kubernetes.io/projected/e3c18953-5efb-4c9b-9e8a-b954fd74863a-kube-api-access-dp5bx\") pod \"kuadrant-operator-catalog-sckj8\" (UID: \"e3c18953-5efb-4c9b-9e8a-b954fd74863a\") " pod="kuadrant-system/kuadrant-operator-catalog-sckj8" Jul 09 03:48:57.766670 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.765963 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sckj8" Jul 09 03:48:57.810672 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.809463 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sckj8"] Jul 09 03:48:57.920917 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:57.920890 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sckj8"] Jul 09 03:48:57.923510 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:48:57.923487 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3c18953_5efb_4c9b_9e8a_b954fd74863a.slice/crio-0f7dcdcf55858d7d2044fc8b9ea5a2bd4d006f05f9ad8cd8cda2f76f37b3f0b0 WatchSource:0}: Error finding container 0f7dcdcf55858d7d2044fc8b9ea5a2bd4d006f05f9ad8cd8cda2f76f37b3f0b0: Status 404 returned error can't find the container with id 0f7dcdcf55858d7d2044fc8b9ea5a2bd4d006f05f9ad8cd8cda2f76f37b3f0b0 Jul 09 03:48:58.016905 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:58.016815 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zp778"] Jul 09 03:48:58.019679 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:58.019636 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zp778" Jul 09 03:48:58.027547 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:58.027509 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zp778"] Jul 09 03:48:58.140462 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:58.140414 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcgwl\" (UniqueName: \"kubernetes.io/projected/0468c160-fd4b-4bdc-8be4-f21218583524-kube-api-access-kcgwl\") pod \"kuadrant-operator-catalog-zp778\" (UID: \"0468c160-fd4b-4bdc-8be4-f21218583524\") " pod="kuadrant-system/kuadrant-operator-catalog-zp778" Jul 09 03:48:58.241444 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:58.241407 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcgwl\" (UniqueName: \"kubernetes.io/projected/0468c160-fd4b-4bdc-8be4-f21218583524-kube-api-access-kcgwl\") pod \"kuadrant-operator-catalog-zp778\" (UID: \"0468c160-fd4b-4bdc-8be4-f21218583524\") " pod="kuadrant-system/kuadrant-operator-catalog-zp778" Jul 09 03:48:58.249266 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:58.249239 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcgwl\" (UniqueName: \"kubernetes.io/projected/0468c160-fd4b-4bdc-8be4-f21218583524-kube-api-access-kcgwl\") pod \"kuadrant-operator-catalog-zp778\" (UID: \"0468c160-fd4b-4bdc-8be4-f21218583524\") " pod="kuadrant-system/kuadrant-operator-catalog-zp778" Jul 09 03:48:58.331770 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:58.331674 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zp778" Jul 09 03:48:58.474261 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:58.474224 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zp778"] Jul 09 03:48:58.477193 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:48:58.477167 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0468c160_fd4b_4bdc_8be4_f21218583524.slice/crio-adadc4cff282e36bc40f85dbe16a21f17532006468314f84609ccb4bc8aa6323 WatchSource:0}: Error finding container adadc4cff282e36bc40f85dbe16a21f17532006468314f84609ccb4bc8aa6323: Status 404 returned error can't find the container with id adadc4cff282e36bc40f85dbe16a21f17532006468314f84609ccb4bc8aa6323 Jul 09 03:48:58.760475 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:58.760434 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sckj8" event={"ID":"e3c18953-5efb-4c9b-9e8a-b954fd74863a","Type":"ContainerStarted","Data":"0f7dcdcf55858d7d2044fc8b9ea5a2bd4d006f05f9ad8cd8cda2f76f37b3f0b0"} Jul 09 03:48:58.761743 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:48:58.761717 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zp778" event={"ID":"0468c160-fd4b-4bdc-8be4-f21218583524","Type":"ContainerStarted","Data":"adadc4cff282e36bc40f85dbe16a21f17532006468314f84609ccb4bc8aa6323"} Jul 09 03:49:00.774510 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:00.774475 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zp778" event={"ID":"0468c160-fd4b-4bdc-8be4-f21218583524","Type":"ContainerStarted","Data":"c37171514670207bbbc91fa008fb71b3e38f08a79255991dc84517bf1d537e89"} Jul 09 03:49:00.775981 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:00.775957 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sckj8" event={"ID":"e3c18953-5efb-4c9b-9e8a-b954fd74863a","Type":"ContainerStarted","Data":"012e7208e789139ecc1c62db9d26e64a2664ac35cf2142937dbc63608d709cb0"} Jul 09 03:49:00.776089 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:00.776042 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-sckj8" podUID="e3c18953-5efb-4c9b-9e8a-b954fd74863a" containerName="registry-server" containerID="cri-o://012e7208e789139ecc1c62db9d26e64a2664ac35cf2142937dbc63608d709cb0" gracePeriod=2 Jul 09 03:49:00.789999 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:00.789949 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-zp778" podStartSLOduration=1.199178562 podStartE2EDuration="2.789932027s" totalImagesPullingTime="1.590753465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:49:00.788958618 +0000 UTC m=+411.420618704" watchObservedRunningTime="2026-07-09 03:49:00.789932027 +0000 UTC m=+411.421592113" Jul 09 03:49:00.803926 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:00.803870 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-sckj8" podStartSLOduration=1.663158783 podStartE2EDuration="3.803853437s" totalImagesPullingTime="2.140694654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:49:00.802115288 +0000 UTC m=+411.433775375" watchObservedRunningTime="2026-07-09 03:49:00.803853437 +0000 UTC m=+411.435513556" Jul 09 03:49:01.029687 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:01.029592 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sckj8" Jul 09 03:49:01.165937 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:01.165895 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp5bx\" (UniqueName: \"kubernetes.io/projected/e3c18953-5efb-4c9b-9e8a-b954fd74863a-kube-api-access-dp5bx\") pod \"e3c18953-5efb-4c9b-9e8a-b954fd74863a\" (UID: \"e3c18953-5efb-4c9b-9e8a-b954fd74863a\") " Jul 09 03:49:01.168196 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:01.168160 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3c18953-5efb-4c9b-9e8a-b954fd74863a-kube-api-access-dp5bx" (OuterVolumeSpecName: "kube-api-access-dp5bx") pod "e3c18953-5efb-4c9b-9e8a-b954fd74863a" (UID: "e3c18953-5efb-4c9b-9e8a-b954fd74863a"). InnerVolumeSpecName "kube-api-access-dp5bx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 03:49:01.267091 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:01.267033 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp5bx\" (UniqueName: \"kubernetes.io/projected/e3c18953-5efb-4c9b-9e8a-b954fd74863a-kube-api-access-dp5bx\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:49:01.781386 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:01.781350 2566 generic.go:356] "Generic (PLEG): container finished" podID="e3c18953-5efb-4c9b-9e8a-b954fd74863a" containerID="012e7208e789139ecc1c62db9d26e64a2664ac35cf2142937dbc63608d709cb0" exitCode=0 Jul 09 03:49:01.781806 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:01.781400 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sckj8" Jul 09 03:49:01.781806 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:01.781434 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sckj8" event={"ID":"e3c18953-5efb-4c9b-9e8a-b954fd74863a","Type":"ContainerDied","Data":"012e7208e789139ecc1c62db9d26e64a2664ac35cf2142937dbc63608d709cb0"} Jul 09 03:49:01.781806 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:01.781498 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sckj8" event={"ID":"e3c18953-5efb-4c9b-9e8a-b954fd74863a","Type":"ContainerDied","Data":"0f7dcdcf55858d7d2044fc8b9ea5a2bd4d006f05f9ad8cd8cda2f76f37b3f0b0"} Jul 09 03:49:01.781806 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:01.781531 2566 scope.go:117] "RemoveContainer" containerID="012e7208e789139ecc1c62db9d26e64a2664ac35cf2142937dbc63608d709cb0" Jul 09 03:49:01.791367 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:01.791345 2566 scope.go:117] "RemoveContainer" containerID="012e7208e789139ecc1c62db9d26e64a2664ac35cf2142937dbc63608d709cb0" Jul 09 03:49:01.791703 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:49:01.791682 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"012e7208e789139ecc1c62db9d26e64a2664ac35cf2142937dbc63608d709cb0\": container with ID starting with 012e7208e789139ecc1c62db9d26e64a2664ac35cf2142937dbc63608d709cb0 not found: ID does not exist" containerID="012e7208e789139ecc1c62db9d26e64a2664ac35cf2142937dbc63608d709cb0" Jul 09 03:49:01.791767 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:01.791745 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"012e7208e789139ecc1c62db9d26e64a2664ac35cf2142937dbc63608d709cb0"} err="failed to get container status \"012e7208e789139ecc1c62db9d26e64a2664ac35cf2142937dbc63608d709cb0\": rpc error: code = NotFound desc = could not find container \"012e7208e789139ecc1c62db9d26e64a2664ac35cf2142937dbc63608d709cb0\": container with ID starting with 012e7208e789139ecc1c62db9d26e64a2664ac35cf2142937dbc63608d709cb0 not found: ID does not exist" Jul 09 03:49:01.802929 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:01.802902 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sckj8"] Jul 09 03:49:01.806998 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:01.806975 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sckj8"] Jul 09 03:49:01.864959 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:01.864916 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3c18953-5efb-4c9b-9e8a-b954fd74863a" path="/var/lib/kubelet/pods/e3c18953-5efb-4c9b-9e8a-b954fd74863a/volumes" Jul 09 03:49:08.332507 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:08.332467 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-zp778" Jul 09 03:49:08.333010 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:08.332535 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-zp778" Jul 09 03:49:08.355689 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:08.355658 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-zp778" Jul 09 03:49:08.836594 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:08.836558 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-zp778" Jul 09 03:49:33.684352 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:33.684268 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-tbh9b"] Jul 09 03:49:33.684866 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:33.684634 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3c18953-5efb-4c9b-9e8a-b954fd74863a" containerName="registry-server" Jul 09 03:49:33.684866 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:33.684659 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3c18953-5efb-4c9b-9e8a-b954fd74863a" containerName="registry-server" Jul 09 03:49:33.684866 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:33.684745 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3c18953-5efb-4c9b-9e8a-b954fd74863a" containerName="registry-server" Jul 09 03:49:33.687703 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:33.687685 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tbh9b" Jul 09 03:49:33.690525 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:33.690500 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 03:49:33.690525 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:33.690516 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-nfgss\"" Jul 09 03:49:33.697038 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:33.697013 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-tbh9b"] Jul 09 03:49:33.728291 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:33.728254 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqnmz\" (UniqueName: \"kubernetes.io/projected/64809dbb-f2d7-4396-b31b-29631d95411b-kube-api-access-wqnmz\") pod \"dns-operator-controller-manager-864dd78c4c-tbh9b\" (UID: \"64809dbb-f2d7-4396-b31b-29631d95411b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tbh9b" Jul 09 03:49:33.829511 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:33.829474 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqnmz\" (UniqueName: \"kubernetes.io/projected/64809dbb-f2d7-4396-b31b-29631d95411b-kube-api-access-wqnmz\") pod \"dns-operator-controller-manager-864dd78c4c-tbh9b\" (UID: \"64809dbb-f2d7-4396-b31b-29631d95411b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tbh9b" Jul 09 03:49:33.844917 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:33.844879 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqnmz\" (UniqueName: \"kubernetes.io/projected/64809dbb-f2d7-4396-b31b-29631d95411b-kube-api-access-wqnmz\") pod \"dns-operator-controller-manager-864dd78c4c-tbh9b\" (UID: \"64809dbb-f2d7-4396-b31b-29631d95411b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tbh9b" Jul 09 03:49:33.998948 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:33.998857 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tbh9b" Jul 09 03:49:34.146618 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:34.146590 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-tbh9b"] Jul 09 03:49:34.149174 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:49:34.149145 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64809dbb_f2d7_4396_b31b_29631d95411b.slice/crio-515e9fc50810d5e435bd5b24f133a1ff83304cea0d46acf7c4c7bffc383efb25 WatchSource:0}: Error finding container 515e9fc50810d5e435bd5b24f133a1ff83304cea0d46acf7c4c7bffc383efb25: Status 404 returned error can't find the container with id 515e9fc50810d5e435bd5b24f133a1ff83304cea0d46acf7c4c7bffc383efb25 Jul 09 03:49:34.941520 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:34.941470 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tbh9b" event={"ID":"64809dbb-f2d7-4396-b31b-29631d95411b","Type":"ContainerStarted","Data":"515e9fc50810d5e435bd5b24f133a1ff83304cea0d46acf7c4c7bffc383efb25"} Jul 09 03:49:36.953070 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:36.953035 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tbh9b" event={"ID":"64809dbb-f2d7-4396-b31b-29631d95411b","Type":"ContainerStarted","Data":"dfec464301150983b72521fddbc517bb3c65d0e1af0dd4773bc255ed4563e746"} Jul 09 03:49:36.953452 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:36.953143 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tbh9b" Jul 09 03:49:36.973391 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:36.973344 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tbh9b" podStartSLOduration=1.9559349639999999 podStartE2EDuration="3.973328069s" totalImagesPullingTime="2.017393105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:49:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:49:36.971215372 +0000 UTC m=+447.602875461" watchObservedRunningTime="2026-07-09 03:49:36.973328069 +0000 UTC m=+447.604988155" Jul 09 03:49:39.012626 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:39.012596 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q"] Jul 09 03:49:39.015885 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:39.015867 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:39.018583 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:39.018561 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fhkmc\"" Jul 09 03:49:39.034064 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:39.034042 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q"] Jul 09 03:49:39.073371 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:39.073337 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2083148-ac6a-4480-a9e9-4f0941e5600d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vbs2q\" (UID: \"f2083148-ac6a-4480-a9e9-4f0941e5600d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:39.073543 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:39.073386 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n2hz\" (UniqueName: \"kubernetes.io/projected/f2083148-ac6a-4480-a9e9-4f0941e5600d-kube-api-access-4n2hz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vbs2q\" (UID: \"f2083148-ac6a-4480-a9e9-4f0941e5600d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:39.174619 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:39.174586 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2083148-ac6a-4480-a9e9-4f0941e5600d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vbs2q\" (UID: \"f2083148-ac6a-4480-a9e9-4f0941e5600d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:39.174851 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:39.174627 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n2hz\" (UniqueName: \"kubernetes.io/projected/f2083148-ac6a-4480-a9e9-4f0941e5600d-kube-api-access-4n2hz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vbs2q\" (UID: \"f2083148-ac6a-4480-a9e9-4f0941e5600d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:39.175163 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:39.175137 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2083148-ac6a-4480-a9e9-4f0941e5600d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vbs2q\" (UID: \"f2083148-ac6a-4480-a9e9-4f0941e5600d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:39.188582 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:39.188557 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n2hz\" (UniqueName: \"kubernetes.io/projected/f2083148-ac6a-4480-a9e9-4f0941e5600d-kube-api-access-4n2hz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vbs2q\" (UID: \"f2083148-ac6a-4480-a9e9-4f0941e5600d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:39.325743 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:39.325624 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:39.464738 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:39.464714 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q"] Jul 09 03:49:39.467403 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:49:39.467382 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2083148_ac6a_4480_a9e9_4f0941e5600d.slice/crio-5ad4f86af4bcf88095275ecd8aa3eda893cde659051e17ee469634e3057615a5 WatchSource:0}: Error finding container 5ad4f86af4bcf88095275ecd8aa3eda893cde659051e17ee469634e3057615a5: Status 404 returned error can't find the container with id 5ad4f86af4bcf88095275ecd8aa3eda893cde659051e17ee469634e3057615a5 Jul 09 03:49:39.968067 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:39.968028 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" event={"ID":"f2083148-ac6a-4480-a9e9-4f0941e5600d","Type":"ContainerStarted","Data":"5ad4f86af4bcf88095275ecd8aa3eda893cde659051e17ee469634e3057615a5"} Jul 09 03:49:42.061196 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:42.061157 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n"] Jul 09 03:49:42.065098 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:42.065076 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" Jul 09 03:49:42.068960 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:42.068938 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-xv98x\"" Jul 09 03:49:42.076376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:42.076354 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n"] Jul 09 03:49:42.201881 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:42.201841 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vvgp\" (UniqueName: \"kubernetes.io/projected/22509125-5078-46e4-a1b2-74db28e4d240-kube-api-access-7vvgp\") pod \"limitador-operator-controller-manager-859d7b78bd-9bv2n\" (UID: \"22509125-5078-46e4-a1b2-74db28e4d240\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" Jul 09 03:49:42.302500 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:42.302456 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vvgp\" (UniqueName: \"kubernetes.io/projected/22509125-5078-46e4-a1b2-74db28e4d240-kube-api-access-7vvgp\") pod \"limitador-operator-controller-manager-859d7b78bd-9bv2n\" (UID: \"22509125-5078-46e4-a1b2-74db28e4d240\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" Jul 09 03:49:42.315038 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:42.314975 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vvgp\" (UniqueName: \"kubernetes.io/projected/22509125-5078-46e4-a1b2-74db28e4d240-kube-api-access-7vvgp\") pod \"limitador-operator-controller-manager-859d7b78bd-9bv2n\" (UID: \"22509125-5078-46e4-a1b2-74db28e4d240\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" Jul 09 03:49:42.379790 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:42.379752 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" Jul 09 03:49:42.532262 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:42.532240 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n"] Jul 09 03:49:42.535113 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:49:42.535085 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22509125_5078_46e4_a1b2_74db28e4d240.slice/crio-1196627a6172e6b0181797637be58b1627b356ae2854ec8d8b383e7a5e97b0cb WatchSource:0}: Error finding container 1196627a6172e6b0181797637be58b1627b356ae2854ec8d8b383e7a5e97b0cb: Status 404 returned error can't find the container with id 1196627a6172e6b0181797637be58b1627b356ae2854ec8d8b383e7a5e97b0cb Jul 09 03:49:42.985573 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:42.985534 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" event={"ID":"22509125-5078-46e4-a1b2-74db28e4d240","Type":"ContainerStarted","Data":"1196627a6172e6b0181797637be58b1627b356ae2854ec8d8b383e7a5e97b0cb"} Jul 09 03:49:46.004305 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:46.004269 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" event={"ID":"22509125-5078-46e4-a1b2-74db28e4d240","Type":"ContainerStarted","Data":"56cc2c94c1f082f49bbdf5e88961964806c789fa68a7096ee9badd1037617020"} Jul 09 03:49:46.004773 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:46.004336 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" Jul 09 03:49:46.005943 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:46.005919 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" event={"ID":"f2083148-ac6a-4480-a9e9-4f0941e5600d","Type":"ContainerStarted","Data":"3faea9725a5b9126aaac905eefde461fa266407ca6f2c565548d2b29af8a58bb"} Jul 09 03:49:46.006026 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:46.005983 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:46.026394 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:46.026344 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" podStartSLOduration=1.365376691 podStartE2EDuration="4.02632765s" totalImagesPullingTime="2.660950959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:49:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:49:46.026281851 +0000 UTC m=+456.657941937" watchObservedRunningTime="2026-07-09 03:49:46.02632765 +0000 UTC m=+456.657987714" Jul 09 03:49:46.055315 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:46.055263 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" podStartSLOduration=2.315150628 podStartE2EDuration="8.055247207s" totalImagesPullingTime="5.740096579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:49:46.052187091 +0000 UTC m=+456.683847177" watchObservedRunningTime="2026-07-09 03:49:46.055247207 +0000 UTC m=+456.686907381" Jul 09 03:49:46.674393 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:46.674355 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zqkrv"] Jul 09 03:49:46.678336 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:46.678302 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zqkrv" Jul 09 03:49:46.681376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:46.681354 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-98ql6\"" Jul 09 03:49:46.690456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:46.690427 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zqkrv"] Jul 09 03:49:46.744796 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:46.744751 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc6bl\" (UniqueName: \"kubernetes.io/projected/f3ca1044-684a-4e0d-ad25-3de28402acf8-kube-api-access-fc6bl\") pod \"authorino-operator-75cc8d5cb7-zqkrv\" (UID: \"f3ca1044-684a-4e0d-ad25-3de28402acf8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zqkrv" Jul 09 03:49:46.846024 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:46.845972 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc6bl\" (UniqueName: \"kubernetes.io/projected/f3ca1044-684a-4e0d-ad25-3de28402acf8-kube-api-access-fc6bl\") pod \"authorino-operator-75cc8d5cb7-zqkrv\" (UID: \"f3ca1044-684a-4e0d-ad25-3de28402acf8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zqkrv" Jul 09 03:49:46.860073 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:46.860041 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc6bl\" (UniqueName: \"kubernetes.io/projected/f3ca1044-684a-4e0d-ad25-3de28402acf8-kube-api-access-fc6bl\") pod \"authorino-operator-75cc8d5cb7-zqkrv\" (UID: \"f3ca1044-684a-4e0d-ad25-3de28402acf8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zqkrv" Jul 09 03:49:46.989550 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:46.989444 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zqkrv" Jul 09 03:49:47.140602 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:47.140574 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zqkrv"] Jul 09 03:49:47.143340 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:49:47.143304 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3ca1044_684a_4e0d_ad25_3de28402acf8.slice/crio-439527151a53d921361734d974edada9cd9d396bc7cb2cbd79eb83c8551ff506 WatchSource:0}: Error finding container 439527151a53d921361734d974edada9cd9d396bc7cb2cbd79eb83c8551ff506: Status 404 returned error can't find the container with id 439527151a53d921361734d974edada9cd9d396bc7cb2cbd79eb83c8551ff506 Jul 09 03:49:47.960690 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:47.960640 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-tbh9b" Jul 09 03:49:48.021343 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:48.021299 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zqkrv" event={"ID":"f3ca1044-684a-4e0d-ad25-3de28402acf8","Type":"ContainerStarted","Data":"439527151a53d921361734d974edada9cd9d396bc7cb2cbd79eb83c8551ff506"} Jul 09 03:49:50.033466 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:50.033429 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zqkrv" event={"ID":"f3ca1044-684a-4e0d-ad25-3de28402acf8","Type":"ContainerStarted","Data":"951bf50e8b53b39d77165041f65e833a3afa4ff140ca8cec3ff1a6ff9695e071"} Jul 09 03:49:50.033875 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:50.033548 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zqkrv" Jul 09 03:49:50.052532 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:50.052481 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zqkrv" podStartSLOduration=1.665386289 podStartE2EDuration="4.052459575s" totalImagesPullingTime="2.387073286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:49:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:49:50.050582304 +0000 UTC m=+460.682242389" watchObservedRunningTime="2026-07-09 03:49:50.052459575 +0000 UTC m=+460.684119662" Jul 09 03:49:57.015009 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:57.014977 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" Jul 09 03:49:57.015379 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:57.015054 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:58.796128 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.796088 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q"] Jul 09 03:49:58.796548 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.796291 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" podUID="f2083148-ac6a-4480-a9e9-4f0941e5600d" containerName="manager" containerID="cri-o://3faea9725a5b9126aaac905eefde461fa266407ca6f2c565548d2b29af8a58bb" gracePeriod=2 Jul 09 03:49:58.798838 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:49:58.798796 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbs2q\" is forbidden: User \"system:node:ip-10-0-137-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-230.ec2.internal' and this object" podUID="f2083148-ac6a-4480-a9e9-4f0941e5600d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:58.799579 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.799551 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q"] Jul 09 03:49:58.819334 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.819306 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp"] Jul 09 03:49:58.819916 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.819896 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2083148-ac6a-4480-a9e9-4f0941e5600d" containerName="manager" Jul 09 03:49:58.819958 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.819921 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2083148-ac6a-4480-a9e9-4f0941e5600d" containerName="manager" Jul 09 03:49:58.820050 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.820038 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2083148-ac6a-4480-a9e9-4f0941e5600d" containerName="manager" Jul 09 03:49:58.823309 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.823282 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" Jul 09 03:49:58.839293 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.839259 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n"] Jul 09 03:49:58.839789 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.839755 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" podUID="22509125-5078-46e4-a1b2-74db28e4d240" containerName="manager" containerID="cri-o://56cc2c94c1f082f49bbdf5e88961964806c789fa68a7096ee9badd1037617020" gracePeriod=2 Jul 09 03:49:58.842082 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.842056 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp"] Jul 09 03:49:58.847162 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.847132 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n"] Jul 09 03:49:58.866019 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:49:58.865981 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbs2q\" is forbidden: User \"system:node:ip-10-0-137-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-230.ec2.internal' and this object" podUID="f2083148-ac6a-4480-a9e9-4f0941e5600d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:58.867773 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.867749 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wt664"] Jul 09 03:49:58.868258 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:49:58.868228 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbs2q\" is forbidden: User \"system:node:ip-10-0-137-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-230.ec2.internal' and this object" podUID="f2083148-ac6a-4480-a9e9-4f0941e5600d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:58.868405 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.868385 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22509125-5078-46e4-a1b2-74db28e4d240" containerName="manager" Jul 09 03:49:58.868459 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.868409 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="22509125-5078-46e4-a1b2-74db28e4d240" containerName="manager" Jul 09 03:49:58.868514 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.868503 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="22509125-5078-46e4-a1b2-74db28e4d240" containerName="manager" Jul 09 03:49:58.870827 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:49:58.870801 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9bv2n\" is forbidden: User \"system:node:ip-10-0-137-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-230.ec2.internal' and this object" podUID="22509125-5078-46e4-a1b2-74db28e4d240" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" Jul 09 03:49:58.871947 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.871930 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wt664" Jul 09 03:49:58.874380 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:49:58.874355 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbs2q\" is forbidden: User \"system:node:ip-10-0-137-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-230.ec2.internal' and this object" podUID="f2083148-ac6a-4480-a9e9-4f0941e5600d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:58.876711 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:49:58.876684 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9bv2n\" is forbidden: User \"system:node:ip-10-0-137-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-230.ec2.internal' and this object" podUID="22509125-5078-46e4-a1b2-74db28e4d240" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" Jul 09 03:49:58.883547 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.883512 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wt664"] Jul 09 03:49:58.955609 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.955571 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2eba061-649d-4082-ba70-86c6e4d382b0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dhltp\" (UID: \"e2eba061-649d-4082-ba70-86c6e4d382b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" Jul 09 03:49:58.955779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.955748 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgkzr\" (UniqueName: \"kubernetes.io/projected/a7bb340b-197e-4341-bf20-5f47299cb3fa-kube-api-access-bgkzr\") pod \"limitador-operator-controller-manager-859d7b78bd-wt664\" (UID: \"a7bb340b-197e-4341-bf20-5f47299cb3fa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wt664" Jul 09 03:49:58.955875 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:58.955854 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msjz5\" (UniqueName: \"kubernetes.io/projected/e2eba061-649d-4082-ba70-86c6e4d382b0-kube-api-access-msjz5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dhltp\" (UID: \"e2eba061-649d-4082-ba70-86c6e4d382b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" Jul 09 03:49:59.057296 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.057200 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2eba061-649d-4082-ba70-86c6e4d382b0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dhltp\" (UID: \"e2eba061-649d-4082-ba70-86c6e4d382b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" Jul 09 03:49:59.057296 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.057270 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgkzr\" (UniqueName: \"kubernetes.io/projected/a7bb340b-197e-4341-bf20-5f47299cb3fa-kube-api-access-bgkzr\") pod \"limitador-operator-controller-manager-859d7b78bd-wt664\" (UID: \"a7bb340b-197e-4341-bf20-5f47299cb3fa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wt664" Jul 09 03:49:59.057516 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.057310 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msjz5\" (UniqueName: \"kubernetes.io/projected/e2eba061-649d-4082-ba70-86c6e4d382b0-kube-api-access-msjz5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dhltp\" (UID: \"e2eba061-649d-4082-ba70-86c6e4d382b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" Jul 09 03:49:59.057628 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.057606 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2eba061-649d-4082-ba70-86c6e4d382b0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dhltp\" (UID: \"e2eba061-649d-4082-ba70-86c6e4d382b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" Jul 09 03:49:59.066868 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.066835 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgkzr\" (UniqueName: \"kubernetes.io/projected/a7bb340b-197e-4341-bf20-5f47299cb3fa-kube-api-access-bgkzr\") pod \"limitador-operator-controller-manager-859d7b78bd-wt664\" (UID: \"a7bb340b-197e-4341-bf20-5f47299cb3fa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wt664" Jul 09 03:49:59.067011 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.066941 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msjz5\" (UniqueName: \"kubernetes.io/projected/e2eba061-649d-4082-ba70-86c6e4d382b0-kube-api-access-msjz5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dhltp\" (UID: \"e2eba061-649d-4082-ba70-86c6e4d382b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" Jul 09 03:49:59.080178 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.080149 2566 generic.go:356] "Generic (PLEG): container finished" podID="f2083148-ac6a-4480-a9e9-4f0941e5600d" containerID="3faea9725a5b9126aaac905eefde461fa266407ca6f2c565548d2b29af8a58bb" exitCode=0 Jul 09 03:49:59.080315 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.080229 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ad4f86af4bcf88095275ecd8aa3eda893cde659051e17ee469634e3057615a5" Jul 09 03:49:59.082119 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.082090 2566 generic.go:356] "Generic (PLEG): container finished" podID="22509125-5078-46e4-a1b2-74db28e4d240" containerID="56cc2c94c1f082f49bbdf5e88961964806c789fa68a7096ee9badd1037617020" exitCode=0 Jul 09 03:49:59.102574 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.102543 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:59.104771 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.104745 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" Jul 09 03:49:59.105189 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:49:59.105161 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbs2q\" is forbidden: User \"system:node:ip-10-0-137-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-230.ec2.internal' and this object" podUID="f2083148-ac6a-4480-a9e9-4f0941e5600d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:59.107460 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:49:59.107430 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9bv2n\" is forbidden: User \"system:node:ip-10-0-137-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-230.ec2.internal' and this object" podUID="22509125-5078-46e4-a1b2-74db28e4d240" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" Jul 09 03:49:59.109455 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:49:59.109434 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbs2q\" is forbidden: User \"system:node:ip-10-0-137-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-230.ec2.internal' and this object" podUID="f2083148-ac6a-4480-a9e9-4f0941e5600d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:59.111789 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:49:59.111768 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9bv2n\" is forbidden: User \"system:node:ip-10-0-137-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-230.ec2.internal' and this object" podUID="22509125-5078-46e4-a1b2-74db28e4d240" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" Jul 09 03:49:59.211954 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.211916 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" Jul 09 03:49:59.217985 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.217954 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wt664" Jul 09 03:49:59.259374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.259268 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4n2hz\" (UniqueName: \"kubernetes.io/projected/f2083148-ac6a-4480-a9e9-4f0941e5600d-kube-api-access-4n2hz\") pod \"f2083148-ac6a-4480-a9e9-4f0941e5600d\" (UID: \"f2083148-ac6a-4480-a9e9-4f0941e5600d\") " Jul 09 03:49:59.259374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.259332 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2083148-ac6a-4480-a9e9-4f0941e5600d-extensions-socket-volume\") pod \"f2083148-ac6a-4480-a9e9-4f0941e5600d\" (UID: \"f2083148-ac6a-4480-a9e9-4f0941e5600d\") " Jul 09 03:49:59.259374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.259373 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vvgp\" (UniqueName: \"kubernetes.io/projected/22509125-5078-46e4-a1b2-74db28e4d240-kube-api-access-7vvgp\") pod \"22509125-5078-46e4-a1b2-74db28e4d240\" (UID: \"22509125-5078-46e4-a1b2-74db28e4d240\") " Jul 09 03:49:59.260191 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.260162 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2083148-ac6a-4480-a9e9-4f0941e5600d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f2083148-ac6a-4480-a9e9-4f0941e5600d" (UID: "f2083148-ac6a-4480-a9e9-4f0941e5600d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 03:49:59.261777 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.261745 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2083148-ac6a-4480-a9e9-4f0941e5600d-kube-api-access-4n2hz" (OuterVolumeSpecName: "kube-api-access-4n2hz") pod "f2083148-ac6a-4480-a9e9-4f0941e5600d" (UID: "f2083148-ac6a-4480-a9e9-4f0941e5600d"). InnerVolumeSpecName "kube-api-access-4n2hz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 03:49:59.262557 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.262517 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22509125-5078-46e4-a1b2-74db28e4d240-kube-api-access-7vvgp" (OuterVolumeSpecName: "kube-api-access-7vvgp") pod "22509125-5078-46e4-a1b2-74db28e4d240" (UID: "22509125-5078-46e4-a1b2-74db28e4d240"). InnerVolumeSpecName "kube-api-access-7vvgp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 03:49:59.360187 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.360156 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n2hz\" (UniqueName: \"kubernetes.io/projected/f2083148-ac6a-4480-a9e9-4f0941e5600d-kube-api-access-4n2hz\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:49:59.360187 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.360186 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2083148-ac6a-4480-a9e9-4f0941e5600d-extensions-socket-volume\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:49:59.360376 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.360200 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vvgp\" (UniqueName: \"kubernetes.io/projected/22509125-5078-46e4-a1b2-74db28e4d240-kube-api-access-7vvgp\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:49:59.381675 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.381617 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp"] Jul 09 03:49:59.385267 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:49:59.385234 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2eba061_649d_4082_ba70_86c6e4d382b0.slice/crio-b1daa06ee0cef8547a1fdcd142af2315ff3a549442ed6fbb86ac553d5c2bd4a1 WatchSource:0}: Error finding container b1daa06ee0cef8547a1fdcd142af2315ff3a549442ed6fbb86ac553d5c2bd4a1: Status 404 returned error can't find the container with id b1daa06ee0cef8547a1fdcd142af2315ff3a549442ed6fbb86ac553d5c2bd4a1 Jul 09 03:49:59.404808 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.404780 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wt664"] Jul 09 03:49:59.409015 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:49:59.408982 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7bb340b_197e_4341_bf20_5f47299cb3fa.slice/crio-796eee8860af7500e6363c7ebd0ad5b2926cc7146a7276065f39d39ede22602b WatchSource:0}: Error finding container 796eee8860af7500e6363c7ebd0ad5b2926cc7146a7276065f39d39ede22602b: Status 404 returned error can't find the container with id 796eee8860af7500e6363c7ebd0ad5b2926cc7146a7276065f39d39ede22602b Jul 09 03:49:59.862576 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:49:59.862537 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbs2q\" is forbidden: User \"system:node:ip-10-0-137-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-230.ec2.internal' and this object" podUID="f2083148-ac6a-4480-a9e9-4f0941e5600d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:49:59.870786 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.864805 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22509125-5078-46e4-a1b2-74db28e4d240" path="/var/lib/kubelet/pods/22509125-5078-46e4-a1b2-74db28e4d240/volumes" Jul 09 03:49:59.870786 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:49:59.864845 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9bv2n\" is forbidden: User \"system:node:ip-10-0-137-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-230.ec2.internal' and this object" podUID="22509125-5078-46e4-a1b2-74db28e4d240" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" Jul 09 03:49:59.870786 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:49:59.865368 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2083148-ac6a-4480-a9e9-4f0941e5600d" path="/var/lib/kubelet/pods/f2083148-ac6a-4480-a9e9-4f0941e5600d/volumes" Jul 09 03:50:00.089579 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:00.089481 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" event={"ID":"e2eba061-649d-4082-ba70-86c6e4d382b0","Type":"ContainerStarted","Data":"30d34731995ebaa412adebc751ab309c16a67ae2bfa614442ef31f5c9b44ce40"} Jul 09 03:50:00.089579 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:00.089550 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" event={"ID":"e2eba061-649d-4082-ba70-86c6e4d382b0","Type":"ContainerStarted","Data":"b1daa06ee0cef8547a1fdcd142af2315ff3a549442ed6fbb86ac553d5c2bd4a1"} Jul 09 03:50:00.089825 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:00.089677 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" Jul 09 03:50:00.091164 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:00.091140 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wt664" event={"ID":"a7bb340b-197e-4341-bf20-5f47299cb3fa","Type":"ContainerStarted","Data":"cb3f5fa238d3fb91f39b0bdfe754d7ced1879bf0856dfbb3a446f6c1745c3797"} Jul 09 03:50:00.091273 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:00.091177 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wt664" event={"ID":"a7bb340b-197e-4341-bf20-5f47299cb3fa","Type":"ContainerStarted","Data":"796eee8860af7500e6363c7ebd0ad5b2926cc7146a7276065f39d39ede22602b"} Jul 09 03:50:00.091273 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:00.091234 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wt664" Jul 09 03:50:00.092489 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:00.092471 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bv2n" Jul 09 03:50:00.092489 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:00.092482 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbs2q" Jul 09 03:50:00.092587 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:00.092499 2566 scope.go:117] "RemoveContainer" containerID="56cc2c94c1f082f49bbdf5e88961964806c789fa68a7096ee9badd1037617020" Jul 09 03:50:00.109691 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:00.109625 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" podStartSLOduration=2.109609964 podStartE2EDuration="2.109609964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:49:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:50:00.108861429 +0000 UTC m=+470.740521515" watchObservedRunningTime="2026-07-09 03:50:00.109609964 +0000 UTC m=+470.741270051" Jul 09 03:50:00.131093 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:00.131036 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wt664" podStartSLOduration=2.131022108 podStartE2EDuration="2.131022108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:49:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:50:00.129828565 +0000 UTC m=+470.761488653" watchObservedRunningTime="2026-07-09 03:50:00.131022108 +0000 UTC m=+470.762682193" Jul 09 03:50:01.041560 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:01.041530 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zqkrv" Jul 09 03:50:11.100514 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:11.100480 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" Jul 09 03:50:11.101029 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:11.100571 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wt664" Jul 09 03:50:15.561003 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:15.560967 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp"] Jul 09 03:50:15.561477 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:15.561157 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" podUID="e2eba061-649d-4082-ba70-86c6e4d382b0" containerName="manager" containerID="cri-o://30d34731995ebaa412adebc751ab309c16a67ae2bfa614442ef31f5c9b44ce40" gracePeriod=10 Jul 09 03:50:15.814415 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:15.814353 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" Jul 09 03:50:15.909946 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:15.909908 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msjz5\" (UniqueName: \"kubernetes.io/projected/e2eba061-649d-4082-ba70-86c6e4d382b0-kube-api-access-msjz5\") pod \"e2eba061-649d-4082-ba70-86c6e4d382b0\" (UID: \"e2eba061-649d-4082-ba70-86c6e4d382b0\") " Jul 09 03:50:15.910117 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:15.910012 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2eba061-649d-4082-ba70-86c6e4d382b0-extensions-socket-volume\") pod \"e2eba061-649d-4082-ba70-86c6e4d382b0\" (UID: \"e2eba061-649d-4082-ba70-86c6e4d382b0\") " Jul 09 03:50:15.910374 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:15.910349 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2eba061-649d-4082-ba70-86c6e4d382b0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e2eba061-649d-4082-ba70-86c6e4d382b0" (UID: "e2eba061-649d-4082-ba70-86c6e4d382b0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 03:50:15.912129 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:15.912100 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2eba061-649d-4082-ba70-86c6e4d382b0-kube-api-access-msjz5" (OuterVolumeSpecName: "kube-api-access-msjz5") pod "e2eba061-649d-4082-ba70-86c6e4d382b0" (UID: "e2eba061-649d-4082-ba70-86c6e4d382b0"). InnerVolumeSpecName "kube-api-access-msjz5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 03:50:16.011460 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:16.011420 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2eba061-649d-4082-ba70-86c6e4d382b0-extensions-socket-volume\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:50:16.011460 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:16.011451 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msjz5\" (UniqueName: \"kubernetes.io/projected/e2eba061-649d-4082-ba70-86c6e4d382b0-kube-api-access-msjz5\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:50:16.172309 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:16.172273 2566 generic.go:356] "Generic (PLEG): container finished" podID="e2eba061-649d-4082-ba70-86c6e4d382b0" containerID="30d34731995ebaa412adebc751ab309c16a67ae2bfa614442ef31f5c9b44ce40" exitCode=0 Jul 09 03:50:16.172515 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:16.172318 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" Jul 09 03:50:16.172515 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:16.172327 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" event={"ID":"e2eba061-649d-4082-ba70-86c6e4d382b0","Type":"ContainerDied","Data":"30d34731995ebaa412adebc751ab309c16a67ae2bfa614442ef31f5c9b44ce40"} Jul 09 03:50:16.172515 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:16.172379 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp" event={"ID":"e2eba061-649d-4082-ba70-86c6e4d382b0","Type":"ContainerDied","Data":"b1daa06ee0cef8547a1fdcd142af2315ff3a549442ed6fbb86ac553d5c2bd4a1"} Jul 09 03:50:16.172515 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:16.172406 2566 scope.go:117] "RemoveContainer" containerID="30d34731995ebaa412adebc751ab309c16a67ae2bfa614442ef31f5c9b44ce40" Jul 09 03:50:16.181931 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:16.181901 2566 scope.go:117] "RemoveContainer" containerID="30d34731995ebaa412adebc751ab309c16a67ae2bfa614442ef31f5c9b44ce40" Jul 09 03:50:16.182209 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:50:16.182193 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d34731995ebaa412adebc751ab309c16a67ae2bfa614442ef31f5c9b44ce40\": container with ID starting with 30d34731995ebaa412adebc751ab309c16a67ae2bfa614442ef31f5c9b44ce40 not found: ID does not exist" containerID="30d34731995ebaa412adebc751ab309c16a67ae2bfa614442ef31f5c9b44ce40" Jul 09 03:50:16.182255 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:16.182218 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d34731995ebaa412adebc751ab309c16a67ae2bfa614442ef31f5c9b44ce40"} err="failed to get container status \"30d34731995ebaa412adebc751ab309c16a67ae2bfa614442ef31f5c9b44ce40\": rpc error: code = NotFound desc = could not find container \"30d34731995ebaa412adebc751ab309c16a67ae2bfa614442ef31f5c9b44ce40\": container with ID starting with 30d34731995ebaa412adebc751ab309c16a67ae2bfa614442ef31f5c9b44ce40 not found: ID does not exist" Jul 09 03:50:16.195898 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:16.195872 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp"] Jul 09 03:50:16.209215 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:16.209178 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dhltp"] Jul 09 03:50:17.865592 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:17.865555 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2eba061-649d-4082-ba70-86c6e4d382b0" path="/var/lib/kubelet/pods/e2eba061-649d-4082-ba70-86c6e4d382b0/volumes" Jul 09 03:50:31.888217 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.888181 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn"] Jul 09 03:50:31.888608 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.888557 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2eba061-649d-4082-ba70-86c6e4d382b0" containerName="manager" Jul 09 03:50:31.888608 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.888567 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2eba061-649d-4082-ba70-86c6e4d382b0" containerName="manager" Jul 09 03:50:31.888722 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.888630 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2eba061-649d-4082-ba70-86c6e4d382b0" containerName="manager" Jul 09 03:50:31.896092 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.896066 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:31.899013 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.898987 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-c6wrp\"" Jul 09 03:50:31.904865 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.904839 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn"] Jul 09 03:50:31.948013 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.947966 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4953493d-d4b5-4068-b41c-488d1aec4e79-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:31.948013 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.948015 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xlct\" (UniqueName: \"kubernetes.io/projected/4953493d-d4b5-4068-b41c-488d1aec4e79-kube-api-access-4xlct\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:31.948253 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.948041 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:31.948253 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.948072 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:31.948253 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.948093 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:31.948253 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.948164 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4953493d-d4b5-4068-b41c-488d1aec4e79-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:31.948253 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.948220 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:31.948437 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.948266 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:31.948437 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:31.948282 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4953493d-d4b5-4068-b41c-488d1aec4e79-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.049240 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.049204 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4953493d-d4b5-4068-b41c-488d1aec4e79-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.049444 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.049245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xlct\" (UniqueName: \"kubernetes.io/projected/4953493d-d4b5-4068-b41c-488d1aec4e79-kube-api-access-4xlct\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.049444 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.049273 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.049444 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.049307 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.049444 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.049331 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.049444 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.049360 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4953493d-d4b5-4068-b41c-488d1aec4e79-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.049736 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.049502 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.049736 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.049634 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.049736 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.049684 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4953493d-d4b5-4068-b41c-488d1aec4e79-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.049896 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.049792 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.049896 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.049837 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.049999 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.049983 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.050050 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.049981 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.050438 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.050412 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4953493d-d4b5-4068-b41c-488d1aec4e79-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.051788 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.051764 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4953493d-d4b5-4068-b41c-488d1aec4e79-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.051994 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.051980 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4953493d-d4b5-4068-b41c-488d1aec4e79-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.056984 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.056959 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4953493d-d4b5-4068-b41c-488d1aec4e79-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.057176 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.057155 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xlct\" (UniqueName: \"kubernetes.io/projected/4953493d-d4b5-4068-b41c-488d1aec4e79-kube-api-access-4xlct\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fw4vn\" (UID: \"4953493d-d4b5-4068-b41c-488d1aec4e79\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.207812 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.207753 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:32.349234 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.349206 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn"] Jul 09 03:50:32.351828 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:50:32.351798 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4953493d_d4b5_4068_b41c_488d1aec4e79.slice/crio-b3b3276c30b15e42e75331f5b223be9357a36a31590ebc1581f72d62547ee2ff WatchSource:0}: Error finding container b3b3276c30b15e42e75331f5b223be9357a36a31590ebc1581f72d62547ee2ff: Status 404 returned error can't find the container with id b3b3276c30b15e42e75331f5b223be9357a36a31590ebc1581f72d62547ee2ff Jul 09 03:50:32.353869 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.353835 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 03:50:32.353958 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.353911 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 03:50:32.353958 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:32.353939 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 03:50:33.266456 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:33.266414 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" event={"ID":"4953493d-d4b5-4068-b41c-488d1aec4e79","Type":"ContainerStarted","Data":"afcc21dde5e75d987143f97dcf1416e7058c106ccd3927546fde94ff7bc35fd1"} Jul 09 03:50:33.266920 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:33.266476 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" event={"ID":"4953493d-d4b5-4068-b41c-488d1aec4e79","Type":"ContainerStarted","Data":"b3b3276c30b15e42e75331f5b223be9357a36a31590ebc1581f72d62547ee2ff"} Jul 09 03:50:33.287984 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:33.287930 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" podStartSLOduration=2.287914549 podStartE2EDuration="2.287914549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:50:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:50:33.286066639 +0000 UTC m=+503.917726726" watchObservedRunningTime="2026-07-09 03:50:33.287914549 +0000 UTC m=+503.919574699" Jul 09 03:50:34.208680 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.208602 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:34.213513 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.213487 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:34.229314 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.229281 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-428n2"] Jul 09 03:50:34.232917 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.232895 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" Jul 09 03:50:34.235638 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.235614 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gkqxk\"" Jul 09 03:50:34.235804 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.235641 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 03:50:34.242032 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.241999 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-428n2"] Jul 09 03:50:34.268891 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.268850 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhpg7\" (UniqueName: \"kubernetes.io/projected/68d1faa3-4df1-40c9-9047-1c7a1dd4f21f-kube-api-access-hhpg7\") pod \"limitador-limitador-64c94fd64f-428n2\" (UID: \"68d1faa3-4df1-40c9-9047-1c7a1dd4f21f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" Jul 09 03:50:34.269279 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.268994 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68d1faa3-4df1-40c9-9047-1c7a1dd4f21f-config-file\") pod \"limitador-limitador-64c94fd64f-428n2\" (UID: \"68d1faa3-4df1-40c9-9047-1c7a1dd4f21f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" Jul 09 03:50:34.271806 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.271771 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:34.272921 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.272897 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:50:34.327586 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.327544 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-428n2"] Jul 09 03:50:34.369725 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.369694 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhpg7\" (UniqueName: \"kubernetes.io/projected/68d1faa3-4df1-40c9-9047-1c7a1dd4f21f-kube-api-access-hhpg7\") pod \"limitador-limitador-64c94fd64f-428n2\" (UID: \"68d1faa3-4df1-40c9-9047-1c7a1dd4f21f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" Jul 09 03:50:34.370057 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.370038 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68d1faa3-4df1-40c9-9047-1c7a1dd4f21f-config-file\") pod \"limitador-limitador-64c94fd64f-428n2\" (UID: \"68d1faa3-4df1-40c9-9047-1c7a1dd4f21f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" Jul 09 03:50:34.370564 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.370544 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68d1faa3-4df1-40c9-9047-1c7a1dd4f21f-config-file\") pod \"limitador-limitador-64c94fd64f-428n2\" (UID: \"68d1faa3-4df1-40c9-9047-1c7a1dd4f21f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" Jul 09 03:50:34.378194 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.378161 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhpg7\" (UniqueName: \"kubernetes.io/projected/68d1faa3-4df1-40c9-9047-1c7a1dd4f21f-kube-api-access-hhpg7\") pod \"limitador-limitador-64c94fd64f-428n2\" (UID: \"68d1faa3-4df1-40c9-9047-1c7a1dd4f21f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" Jul 09 03:50:34.551550 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.551458 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" Jul 09 03:50:34.692562 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:34.692537 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-428n2"] Jul 09 03:50:34.694365 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:50:34.694334 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68d1faa3_4df1_40c9_9047_1c7a1dd4f21f.slice/crio-03d6702273c166b79e285b50c3192303ddcc4d4672aa9c3320b6102d2b81ad58 WatchSource:0}: Error finding container 03d6702273c166b79e285b50c3192303ddcc4d4672aa9c3320b6102d2b81ad58: Status 404 returned error can't find the container with id 03d6702273c166b79e285b50c3192303ddcc4d4672aa9c3320b6102d2b81ad58 Jul 09 03:50:35.278004 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:35.277965 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" event={"ID":"68d1faa3-4df1-40c9-9047-1c7a1dd4f21f","Type":"ContainerStarted","Data":"03d6702273c166b79e285b50c3192303ddcc4d4672aa9c3320b6102d2b81ad58"} Jul 09 03:50:38.295563 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:38.295524 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" event={"ID":"68d1faa3-4df1-40c9-9047-1c7a1dd4f21f","Type":"ContainerStarted","Data":"4c3371a1f277623abdabfaec21640fd263589579bae16140f0529f615c5d3e21"} Jul 09 03:50:38.296123 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:38.295639 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" Jul 09 03:50:38.310595 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:38.310542 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" podStartSLOduration=1.5551472899999998 podStartE2EDuration="4.310501593s" totalImagesPullingTime="2.755354303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:50:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:50:38.310159606 +0000 UTC m=+508.941819693" watchObservedRunningTime="2026-07-09 03:50:38.310501593 +0000 UTC m=+508.942161679" Jul 09 03:50:48.954053 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:48.954015 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-428n2"] Jul 09 03:50:48.954891 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:48.954254 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" podUID="68d1faa3-4df1-40c9-9047-1c7a1dd4f21f" containerName="limitador" containerID="cri-o://4c3371a1f277623abdabfaec21640fd263589579bae16140f0529f615c5d3e21" gracePeriod=30 Jul 09 03:50:48.955275 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:48.955160 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" Jul 09 03:50:49.358852 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:49.358821 2566 generic.go:356] "Generic (PLEG): container finished" podID="68d1faa3-4df1-40c9-9047-1c7a1dd4f21f" containerID="4c3371a1f277623abdabfaec21640fd263589579bae16140f0529f615c5d3e21" exitCode=0 Jul 09 03:50:49.359030 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:49.358910 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" event={"ID":"68d1faa3-4df1-40c9-9047-1c7a1dd4f21f","Type":"ContainerDied","Data":"4c3371a1f277623abdabfaec21640fd263589579bae16140f0529f615c5d3e21"} Jul 09 03:50:49.907515 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:49.907487 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" Jul 09 03:50:50.014483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.014452 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhpg7\" (UniqueName: \"kubernetes.io/projected/68d1faa3-4df1-40c9-9047-1c7a1dd4f21f-kube-api-access-hhpg7\") pod \"68d1faa3-4df1-40c9-9047-1c7a1dd4f21f\" (UID: \"68d1faa3-4df1-40c9-9047-1c7a1dd4f21f\") " Jul 09 03:50:50.014930 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.014588 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68d1faa3-4df1-40c9-9047-1c7a1dd4f21f-config-file\") pod \"68d1faa3-4df1-40c9-9047-1c7a1dd4f21f\" (UID: \"68d1faa3-4df1-40c9-9047-1c7a1dd4f21f\") " Jul 09 03:50:50.015055 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.015024 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68d1faa3-4df1-40c9-9047-1c7a1dd4f21f-config-file" (OuterVolumeSpecName: "config-file") pod "68d1faa3-4df1-40c9-9047-1c7a1dd4f21f" (UID: "68d1faa3-4df1-40c9-9047-1c7a1dd4f21f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 03:50:50.016846 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.016814 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68d1faa3-4df1-40c9-9047-1c7a1dd4f21f-kube-api-access-hhpg7" (OuterVolumeSpecName: "kube-api-access-hhpg7") pod "68d1faa3-4df1-40c9-9047-1c7a1dd4f21f" (UID: "68d1faa3-4df1-40c9-9047-1c7a1dd4f21f"). InnerVolumeSpecName "kube-api-access-hhpg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 03:50:50.115434 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.115396 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhpg7\" (UniqueName: \"kubernetes.io/projected/68d1faa3-4df1-40c9-9047-1c7a1dd4f21f-kube-api-access-hhpg7\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:50:50.115434 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.115429 2566 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68d1faa3-4df1-40c9-9047-1c7a1dd4f21f-config-file\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:50:50.365969 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.365887 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" Jul 09 03:50:50.366124 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.365884 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-428n2" event={"ID":"68d1faa3-4df1-40c9-9047-1c7a1dd4f21f","Type":"ContainerDied","Data":"03d6702273c166b79e285b50c3192303ddcc4d4672aa9c3320b6102d2b81ad58"} Jul 09 03:50:50.366124 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.366034 2566 scope.go:117] "RemoveContainer" containerID="4c3371a1f277623abdabfaec21640fd263589579bae16140f0529f615c5d3e21" Jul 09 03:50:50.389831 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.389796 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-428n2"] Jul 09 03:50:50.394183 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.394152 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-428n2"] Jul 09 03:50:50.770314 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.770273 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5"] Jul 09 03:50:50.770729 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.770716 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68d1faa3-4df1-40c9-9047-1c7a1dd4f21f" containerName="limitador" Jul 09 03:50:50.770779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.770732 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="68d1faa3-4df1-40c9-9047-1c7a1dd4f21f" containerName="limitador" Jul 09 03:50:50.770830 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.770820 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="68d1faa3-4df1-40c9-9047-1c7a1dd4f21f" containerName="limitador" Jul 09 03:50:50.775437 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.775414 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5" Jul 09 03:50:50.778100 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.778067 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 03:50:50.778100 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.778067 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-8x6l7\"" Jul 09 03:50:50.778100 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.778103 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 09 03:50:50.778331 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.778080 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 03:50:50.782206 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.782114 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5"] Jul 09 03:50:50.923642 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.923604 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/54ae7687-a094-4ff5-917f-34784e5bd3b4-data\") pod \"postgres-69fb4594cf-fwqt5\" (UID: \"54ae7687-a094-4ff5-917f-34784e5bd3b4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5" Jul 09 03:50:50.923815 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:50.923788 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9zct\" (UniqueName: \"kubernetes.io/projected/54ae7687-a094-4ff5-917f-34784e5bd3b4-kube-api-access-g9zct\") pod \"postgres-69fb4594cf-fwqt5\" (UID: \"54ae7687-a094-4ff5-917f-34784e5bd3b4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5" Jul 09 03:50:51.025048 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:51.024964 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9zct\" (UniqueName: \"kubernetes.io/projected/54ae7687-a094-4ff5-917f-34784e5bd3b4-kube-api-access-g9zct\") pod \"postgres-69fb4594cf-fwqt5\" (UID: \"54ae7687-a094-4ff5-917f-34784e5bd3b4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5" Jul 09 03:50:51.025048 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:51.025015 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/54ae7687-a094-4ff5-917f-34784e5bd3b4-data\") pod \"postgres-69fb4594cf-fwqt5\" (UID: \"54ae7687-a094-4ff5-917f-34784e5bd3b4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5" Jul 09 03:50:51.025427 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:51.025355 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/54ae7687-a094-4ff5-917f-34784e5bd3b4-data\") pod \"postgres-69fb4594cf-fwqt5\" (UID: \"54ae7687-a094-4ff5-917f-34784e5bd3b4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5" Jul 09 03:50:51.034040 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:51.034007 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9zct\" (UniqueName: \"kubernetes.io/projected/54ae7687-a094-4ff5-917f-34784e5bd3b4-kube-api-access-g9zct\") pod \"postgres-69fb4594cf-fwqt5\" (UID: \"54ae7687-a094-4ff5-917f-34784e5bd3b4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5" Jul 09 03:50:51.088670 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:51.088607 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5" Jul 09 03:50:51.226427 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:51.225508 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5"] Jul 09 03:50:51.372739 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:51.372635 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5" event={"ID":"54ae7687-a094-4ff5-917f-34784e5bd3b4","Type":"ContainerStarted","Data":"f0e267fe3dc2ba5bbd596a21000eceb5bae54c1113e90891a83ae530392bfc61"} Jul 09 03:50:51.868026 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:51.867994 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68d1faa3-4df1-40c9-9047-1c7a1dd4f21f" path="/var/lib/kubelet/pods/68d1faa3-4df1-40c9-9047-1c7a1dd4f21f/volumes" Jul 09 03:50:56.407831 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:56.407801 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5" event={"ID":"54ae7687-a094-4ff5-917f-34784e5bd3b4","Type":"ContainerStarted","Data":"13c50040e008779a50080ce070b35479ef15f7e3c9860cfd7fc15a9e2e917fc2"} Jul 09 03:50:56.408192 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:56.407866 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5" Jul 09 03:50:56.425222 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:50:56.425166 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5" podStartSLOduration=1.663369281 podStartE2EDuration="6.425149738s" totalImagesPullingTime="4.761780457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:50:56.42214178 +0000 UTC m=+527.053801869" watchObservedRunningTime="2026-07-09 03:50:56.425149738 +0000 UTC m=+527.056809823" Jul 09 03:51:02.444790 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:02.444705 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fwqt5" Jul 09 03:51:09.011085 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.011049 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-748994945c-67hft"] Jul 09 03:51:09.033364 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.033335 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-748994945c-67hft"] Jul 09 03:51:09.033524 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.033443 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-748994945c-67hft" Jul 09 03:51:09.036347 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.036323 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qvsk5\"" Jul 09 03:51:09.036544 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.036531 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 03:51:09.036605 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.036568 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 03:51:09.171878 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.171837 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-869986db6b-ptn66"] Jul 09 03:51:09.175598 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.175578 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869986db6b-ptn66" Jul 09 03:51:09.186715 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.186683 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869986db6b-ptn66"] Jul 09 03:51:09.194382 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.194352 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19acaca-ca05-4e03-ba07-d1d1133855c7-webhook-cert\") pod \"maas-controller-748994945c-67hft\" (UID: \"e19acaca-ca05-4e03-ba07-d1d1133855c7\") " pod="opendatahub/maas-controller-748994945c-67hft" Jul 09 03:51:09.194541 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.194415 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msxpz\" (UniqueName: \"kubernetes.io/projected/e19acaca-ca05-4e03-ba07-d1d1133855c7-kube-api-access-msxpz\") pod \"maas-controller-748994945c-67hft\" (UID: \"e19acaca-ca05-4e03-ba07-d1d1133855c7\") " pod="opendatahub/maas-controller-748994945c-67hft" Jul 09 03:51:09.295745 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.295608 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msxpz\" (UniqueName: \"kubernetes.io/projected/e19acaca-ca05-4e03-ba07-d1d1133855c7-kube-api-access-msxpz\") pod \"maas-controller-748994945c-67hft\" (UID: \"e19acaca-ca05-4e03-ba07-d1d1133855c7\") " pod="opendatahub/maas-controller-748994945c-67hft" Jul 09 03:51:09.295929 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.295738 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19acaca-ca05-4e03-ba07-d1d1133855c7-webhook-cert\") pod \"maas-controller-748994945c-67hft\" (UID: \"e19acaca-ca05-4e03-ba07-d1d1133855c7\") " pod="opendatahub/maas-controller-748994945c-67hft" Jul 09 03:51:09.295929 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.295814 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt46t\" (UniqueName: \"kubernetes.io/projected/7c04e044-0ac5-4684-bb47-6f6762adea36-kube-api-access-zt46t\") pod \"maas-controller-869986db6b-ptn66\" (UID: \"7c04e044-0ac5-4684-bb47-6f6762adea36\") " pod="opendatahub/maas-controller-869986db6b-ptn66" Jul 09 03:51:09.295929 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.295851 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c04e044-0ac5-4684-bb47-6f6762adea36-webhook-cert\") pod \"maas-controller-869986db6b-ptn66\" (UID: \"7c04e044-0ac5-4684-bb47-6f6762adea36\") " pod="opendatahub/maas-controller-869986db6b-ptn66" Jul 09 03:51:09.295929 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:51:09.295911 2566 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 09 03:51:09.296197 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:51:09.295971 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e19acaca-ca05-4e03-ba07-d1d1133855c7-webhook-cert podName:e19acaca-ca05-4e03-ba07-d1d1133855c7 nodeName:}" failed. No retries permitted until 2026-07-09 03:51:09.795951893 +0000 UTC m=+540.427611964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/e19acaca-ca05-4e03-ba07-d1d1133855c7-webhook-cert") pod "maas-controller-748994945c-67hft" (UID: "e19acaca-ca05-4e03-ba07-d1d1133855c7") : secret "maas-controller-webhook-cert" not found Jul 09 03:51:09.306527 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.306496 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msxpz\" (UniqueName: \"kubernetes.io/projected/e19acaca-ca05-4e03-ba07-d1d1133855c7-kube-api-access-msxpz\") pod \"maas-controller-748994945c-67hft\" (UID: \"e19acaca-ca05-4e03-ba07-d1d1133855c7\") " pod="opendatahub/maas-controller-748994945c-67hft" Jul 09 03:51:09.397449 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.397404 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt46t\" (UniqueName: \"kubernetes.io/projected/7c04e044-0ac5-4684-bb47-6f6762adea36-kube-api-access-zt46t\") pod \"maas-controller-869986db6b-ptn66\" (UID: \"7c04e044-0ac5-4684-bb47-6f6762adea36\") " pod="opendatahub/maas-controller-869986db6b-ptn66" Jul 09 03:51:09.397706 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.397475 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c04e044-0ac5-4684-bb47-6f6762adea36-webhook-cert\") pod \"maas-controller-869986db6b-ptn66\" (UID: \"7c04e044-0ac5-4684-bb47-6f6762adea36\") " pod="opendatahub/maas-controller-869986db6b-ptn66" Jul 09 03:51:09.397706 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:51:09.397622 2566 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 09 03:51:09.397849 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:51:09.397727 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c04e044-0ac5-4684-bb47-6f6762adea36-webhook-cert podName:7c04e044-0ac5-4684-bb47-6f6762adea36 nodeName:}" failed. No retries permitted until 2026-07-09 03:51:09.897710531 +0000 UTC m=+540.529370595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/7c04e044-0ac5-4684-bb47-6f6762adea36-webhook-cert") pod "maas-controller-869986db6b-ptn66" (UID: "7c04e044-0ac5-4684-bb47-6f6762adea36") : secret "maas-controller-webhook-cert" not found Jul 09 03:51:09.407059 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.407015 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt46t\" (UniqueName: \"kubernetes.io/projected/7c04e044-0ac5-4684-bb47-6f6762adea36-kube-api-access-zt46t\") pod \"maas-controller-869986db6b-ptn66\" (UID: \"7c04e044-0ac5-4684-bb47-6f6762adea36\") " pod="opendatahub/maas-controller-869986db6b-ptn66" Jul 09 03:51:09.801280 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.801229 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19acaca-ca05-4e03-ba07-d1d1133855c7-webhook-cert\") pod \"maas-controller-748994945c-67hft\" (UID: \"e19acaca-ca05-4e03-ba07-d1d1133855c7\") " pod="opendatahub/maas-controller-748994945c-67hft" Jul 09 03:51:09.803767 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.803731 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19acaca-ca05-4e03-ba07-d1d1133855c7-webhook-cert\") pod \"maas-controller-748994945c-67hft\" (UID: \"e19acaca-ca05-4e03-ba07-d1d1133855c7\") " pod="opendatahub/maas-controller-748994945c-67hft" Jul 09 03:51:09.902178 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.902135 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c04e044-0ac5-4684-bb47-6f6762adea36-webhook-cert\") pod \"maas-controller-869986db6b-ptn66\" (UID: \"7c04e044-0ac5-4684-bb47-6f6762adea36\") " pod="opendatahub/maas-controller-869986db6b-ptn66" Jul 09 03:51:09.904715 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.904691 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c04e044-0ac5-4684-bb47-6f6762adea36-webhook-cert\") pod \"maas-controller-869986db6b-ptn66\" (UID: \"7c04e044-0ac5-4684-bb47-6f6762adea36\") " pod="opendatahub/maas-controller-869986db6b-ptn66" Jul 09 03:51:09.950436 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.950403 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qvsk5\"" Jul 09 03:51:09.957658 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:09.957624 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-748994945c-67hft" Jul 09 03:51:10.088447 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:10.088416 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869986db6b-ptn66" Jul 09 03:51:10.105081 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:10.105055 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-748994945c-67hft"] Jul 09 03:51:10.107128 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:51:10.107099 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode19acaca_ca05_4e03_ba07_d1d1133855c7.slice/crio-3fc89e8e535d8c9a5a40b4ea51d97285e5967569f63690a0ee2aa1ae97cbf31f WatchSource:0}: Error finding container 3fc89e8e535d8c9a5a40b4ea51d97285e5967569f63690a0ee2aa1ae97cbf31f: Status 404 returned error can't find the container with id 3fc89e8e535d8c9a5a40b4ea51d97285e5967569f63690a0ee2aa1ae97cbf31f Jul 09 03:51:10.232785 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:10.232746 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869986db6b-ptn66"] Jul 09 03:51:10.235760 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:51:10.235730 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c04e044_0ac5_4684_bb47_6f6762adea36.slice/crio-18c45849f1c9850c04fe7bf6c3d272023e40e1148946e421fcd9e76f85804bf1 WatchSource:0}: Error finding container 18c45849f1c9850c04fe7bf6c3d272023e40e1148946e421fcd9e76f85804bf1: Status 404 returned error can't find the container with id 18c45849f1c9850c04fe7bf6c3d272023e40e1148946e421fcd9e76f85804bf1 Jul 09 03:51:10.480470 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:10.480434 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-748994945c-67hft" event={"ID":"e19acaca-ca05-4e03-ba07-d1d1133855c7","Type":"ContainerStarted","Data":"3fc89e8e535d8c9a5a40b4ea51d97285e5967569f63690a0ee2aa1ae97cbf31f"} Jul 09 03:51:10.481512 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:10.481483 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869986db6b-ptn66" event={"ID":"7c04e044-0ac5-4684-bb47-6f6762adea36","Type":"ContainerStarted","Data":"18c45849f1c9850c04fe7bf6c3d272023e40e1148946e421fcd9e76f85804bf1"} Jul 09 03:51:12.862140 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:12.862109 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 03:51:13.502549 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:13.502503 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869986db6b-ptn66" event={"ID":"7c04e044-0ac5-4684-bb47-6f6762adea36","Type":"ContainerStarted","Data":"a83e5aa8df71c75b565fd3251022c070e3dfa5da461c8e4a29a6c8cdd2c8a861"} Jul 09 03:51:13.502800 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:13.502573 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-869986db6b-ptn66" Jul 09 03:51:13.504155 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:13.504124 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-748994945c-67hft" event={"ID":"e19acaca-ca05-4e03-ba07-d1d1133855c7","Type":"ContainerStarted","Data":"65912de62f1715d1808685edde9c2a31bdfb74ae3991de8ca73a10fd5a23b762"} Jul 09 03:51:13.504305 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:13.504186 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-748994945c-67hft" Jul 09 03:51:13.530724 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:13.530671 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-869986db6b-ptn66" podStartSLOduration=1.901638459 podStartE2EDuration="4.530637744s" totalImagesPullingTime="2.628999285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:51:13.529054091 +0000 UTC m=+544.160714178" watchObservedRunningTime="2026-07-09 03:51:13.530637744 +0000 UTC m=+544.162297855" Jul 09 03:51:13.564941 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:13.564881 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-748994945c-67hft" podStartSLOduration=2.815036317 podStartE2EDuration="5.564860904s" totalImagesPullingTime="2.749824587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:51:13.562733734 +0000 UTC m=+544.194393820" watchObservedRunningTime="2026-07-09 03:51:13.564860904 +0000 UTC m=+544.196520991" Jul 09 03:51:24.514740 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:24.514705 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-748994945c-67hft" Jul 09 03:51:24.515251 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:24.514792 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-869986db6b-ptn66" Jul 09 03:51:24.574754 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:24.574710 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-748994945c-67hft"] Jul 09 03:51:24.577387 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:24.577323 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-748994945c-67hft" podUID="e19acaca-ca05-4e03-ba07-d1d1133855c7" containerName="manager" containerID="cri-o://65912de62f1715d1808685edde9c2a31bdfb74ae3991de8ca73a10fd5a23b762" gracePeriod=30 Jul 09 03:51:24.823483 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:24.823451 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-748994945c-67hft" Jul 09 03:51:24.933068 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:24.933029 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19acaca-ca05-4e03-ba07-d1d1133855c7-webhook-cert\") pod \"e19acaca-ca05-4e03-ba07-d1d1133855c7\" (UID: \"e19acaca-ca05-4e03-ba07-d1d1133855c7\") " Jul 09 03:51:24.933068 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:24.933068 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msxpz\" (UniqueName: \"kubernetes.io/projected/e19acaca-ca05-4e03-ba07-d1d1133855c7-kube-api-access-msxpz\") pod \"e19acaca-ca05-4e03-ba07-d1d1133855c7\" (UID: \"e19acaca-ca05-4e03-ba07-d1d1133855c7\") " Jul 09 03:51:24.935411 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:24.935369 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e19acaca-ca05-4e03-ba07-d1d1133855c7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e19acaca-ca05-4e03-ba07-d1d1133855c7" (UID: "e19acaca-ca05-4e03-ba07-d1d1133855c7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 03:51:24.935411 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:24.935388 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e19acaca-ca05-4e03-ba07-d1d1133855c7-kube-api-access-msxpz" (OuterVolumeSpecName: "kube-api-access-msxpz") pod "e19acaca-ca05-4e03-ba07-d1d1133855c7" (UID: "e19acaca-ca05-4e03-ba07-d1d1133855c7"). InnerVolumeSpecName "kube-api-access-msxpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 03:51:25.034127 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:25.034084 2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19acaca-ca05-4e03-ba07-d1d1133855c7-webhook-cert\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:51:25.034127 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:25.034117 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msxpz\" (UniqueName: \"kubernetes.io/projected/e19acaca-ca05-4e03-ba07-d1d1133855c7-kube-api-access-msxpz\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:51:25.583521 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:25.583488 2566 generic.go:356] "Generic (PLEG): container finished" podID="e19acaca-ca05-4e03-ba07-d1d1133855c7" containerID="65912de62f1715d1808685edde9c2a31bdfb74ae3991de8ca73a10fd5a23b762" exitCode=0 Jul 09 03:51:25.583946 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:25.583534 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-748994945c-67hft" Jul 09 03:51:25.583946 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:25.583581 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-748994945c-67hft" event={"ID":"e19acaca-ca05-4e03-ba07-d1d1133855c7","Type":"ContainerDied","Data":"65912de62f1715d1808685edde9c2a31bdfb74ae3991de8ca73a10fd5a23b762"} Jul 09 03:51:25.583946 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:25.583628 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-748994945c-67hft" event={"ID":"e19acaca-ca05-4e03-ba07-d1d1133855c7","Type":"ContainerDied","Data":"3fc89e8e535d8c9a5a40b4ea51d97285e5967569f63690a0ee2aa1ae97cbf31f"} Jul 09 03:51:25.583946 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:25.583673 2566 scope.go:117] "RemoveContainer" containerID="65912de62f1715d1808685edde9c2a31bdfb74ae3991de8ca73a10fd5a23b762" Jul 09 03:51:25.592949 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:25.592932 2566 scope.go:117] "RemoveContainer" containerID="65912de62f1715d1808685edde9c2a31bdfb74ae3991de8ca73a10fd5a23b762" Jul 09 03:51:25.593202 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:51:25.593183 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65912de62f1715d1808685edde9c2a31bdfb74ae3991de8ca73a10fd5a23b762\": container with ID starting with 65912de62f1715d1808685edde9c2a31bdfb74ae3991de8ca73a10fd5a23b762 not found: ID does not exist" containerID="65912de62f1715d1808685edde9c2a31bdfb74ae3991de8ca73a10fd5a23b762" Jul 09 03:51:25.593271 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:25.593210 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65912de62f1715d1808685edde9c2a31bdfb74ae3991de8ca73a10fd5a23b762"} err="failed to get container status \"65912de62f1715d1808685edde9c2a31bdfb74ae3991de8ca73a10fd5a23b762\": rpc error: code = NotFound desc = could not find container \"65912de62f1715d1808685edde9c2a31bdfb74ae3991de8ca73a10fd5a23b762\": container with ID starting with 65912de62f1715d1808685edde9c2a31bdfb74ae3991de8ca73a10fd5a23b762 not found: ID does not exist" Jul 09 03:51:25.605244 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:25.605212 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-748994945c-67hft"] Jul 09 03:51:25.609575 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:25.609542 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-748994945c-67hft"] Jul 09 03:51:25.865198 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:25.865120 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e19acaca-ca05-4e03-ba07-d1d1133855c7" path="/var/lib/kubelet/pods/e19acaca-ca05-4e03-ba07-d1d1133855c7/volumes" Jul 09 03:51:45.304514 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.304468 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h"] Jul 09 03:51:45.305128 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.305112 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e19acaca-ca05-4e03-ba07-d1d1133855c7" containerName="manager" Jul 09 03:51:45.305178 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.305131 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e19acaca-ca05-4e03-ba07-d1d1133855c7" containerName="manager" Jul 09 03:51:45.305229 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.305218 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e19acaca-ca05-4e03-ba07-d1d1133855c7" containerName="manager" Jul 09 03:51:45.308798 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.308776 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" Jul 09 03:51:45.311522 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.311501 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 09 03:51:45.311643 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.311543 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8kmjr\"" Jul 09 03:51:45.317832 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.317806 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h"] Jul 09 03:51:45.420920 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.420879 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73d90d29-b819-4adc-9d92-71611d77ec8c-openshift-service-ca\") pod \"maas-api-7686dfb796-b4p2h\" (UID: \"73d90d29-b819-4adc-9d92-71611d77ec8c\") " pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" Jul 09 03:51:45.421099 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.420985 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4lxn\" (UniqueName: \"kubernetes.io/projected/73d90d29-b819-4adc-9d92-71611d77ec8c-kube-api-access-h4lxn\") pod \"maas-api-7686dfb796-b4p2h\" (UID: \"73d90d29-b819-4adc-9d92-71611d77ec8c\") " pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" Jul 09 03:51:45.421099 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.421012 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73d90d29-b819-4adc-9d92-71611d77ec8c-maas-api-tls\") pod \"maas-api-7686dfb796-b4p2h\" (UID: \"73d90d29-b819-4adc-9d92-71611d77ec8c\") " pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" Jul 09 03:51:45.522056 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.522016 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73d90d29-b819-4adc-9d92-71611d77ec8c-openshift-service-ca\") pod \"maas-api-7686dfb796-b4p2h\" (UID: \"73d90d29-b819-4adc-9d92-71611d77ec8c\") " pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" Jul 09 03:51:45.522225 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.522080 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4lxn\" (UniqueName: \"kubernetes.io/projected/73d90d29-b819-4adc-9d92-71611d77ec8c-kube-api-access-h4lxn\") pod \"maas-api-7686dfb796-b4p2h\" (UID: \"73d90d29-b819-4adc-9d92-71611d77ec8c\") " pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" Jul 09 03:51:45.522225 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.522105 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73d90d29-b819-4adc-9d92-71611d77ec8c-maas-api-tls\") pod \"maas-api-7686dfb796-b4p2h\" (UID: \"73d90d29-b819-4adc-9d92-71611d77ec8c\") " pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" Jul 09 03:51:45.522225 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:51:45.522217 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 09 03:51:45.522385 ip-10-0-137-230 kubenswrapper[2566]: E0709 03:51:45.522367 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73d90d29-b819-4adc-9d92-71611d77ec8c-maas-api-tls podName:73d90d29-b819-4adc-9d92-71611d77ec8c nodeName:}" failed. No retries permitted until 2026-07-09 03:51:46.022340705 +0000 UTC m=+576.654000777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/73d90d29-b819-4adc-9d92-71611d77ec8c-maas-api-tls") pod "maas-api-7686dfb796-b4p2h" (UID: "73d90d29-b819-4adc-9d92-71611d77ec8c") : secret "maas-api-serving-cert" not found Jul 09 03:51:45.523528 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.523494 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73d90d29-b819-4adc-9d92-71611d77ec8c-openshift-service-ca\") pod \"maas-api-7686dfb796-b4p2h\" (UID: \"73d90d29-b819-4adc-9d92-71611d77ec8c\") " pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" Jul 09 03:51:45.532868 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.532838 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4lxn\" (UniqueName: \"kubernetes.io/projected/73d90d29-b819-4adc-9d92-71611d77ec8c-kube-api-access-h4lxn\") pod \"maas-api-7686dfb796-b4p2h\" (UID: \"73d90d29-b819-4adc-9d92-71611d77ec8c\") " pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" Jul 09 03:51:45.802479 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.802444 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-jbk6d"] Jul 09 03:51:45.807749 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.807724 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-jbk6d" Jul 09 03:51:45.819252 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.819221 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-jbk6d"] Jul 09 03:51:45.819721 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.819698 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-5hktm\"" Jul 09 03:51:45.820144 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.820128 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 03:51:45.927316 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.927274 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h77gl\" (UniqueName: \"kubernetes.io/projected/c084818f-bc07-4c5e-a79c-f7b44e240330-kube-api-access-h77gl\") pod \"payload-processing-58f86c7798-jbk6d\" (UID: \"c084818f-bc07-4c5e-a79c-f7b44e240330\") " pod="openshift-ingress/payload-processing-58f86c7798-jbk6d" Jul 09 03:51:45.927517 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:45.927449 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c084818f-bc07-4c5e-a79c-f7b44e240330-plugins-config-volume\") pod \"payload-processing-58f86c7798-jbk6d\" (UID: \"c084818f-bc07-4c5e-a79c-f7b44e240330\") " pod="openshift-ingress/payload-processing-58f86c7798-jbk6d" Jul 09 03:51:46.028159 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:46.028124 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h77gl\" (UniqueName: \"kubernetes.io/projected/c084818f-bc07-4c5e-a79c-f7b44e240330-kube-api-access-h77gl\") pod \"payload-processing-58f86c7798-jbk6d\" (UID: \"c084818f-bc07-4c5e-a79c-f7b44e240330\") " pod="openshift-ingress/payload-processing-58f86c7798-jbk6d" Jul 09 03:51:46.028348 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:46.028174 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73d90d29-b819-4adc-9d92-71611d77ec8c-maas-api-tls\") pod \"maas-api-7686dfb796-b4p2h\" (UID: \"73d90d29-b819-4adc-9d92-71611d77ec8c\") " pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" Jul 09 03:51:46.028348 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:46.028215 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c084818f-bc07-4c5e-a79c-f7b44e240330-plugins-config-volume\") pod \"payload-processing-58f86c7798-jbk6d\" (UID: \"c084818f-bc07-4c5e-a79c-f7b44e240330\") " pod="openshift-ingress/payload-processing-58f86c7798-jbk6d" Jul 09 03:51:46.029136 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:46.029111 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c084818f-bc07-4c5e-a79c-f7b44e240330-plugins-config-volume\") pod \"payload-processing-58f86c7798-jbk6d\" (UID: \"c084818f-bc07-4c5e-a79c-f7b44e240330\") " pod="openshift-ingress/payload-processing-58f86c7798-jbk6d" Jul 09 03:51:46.030839 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:46.030819 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73d90d29-b819-4adc-9d92-71611d77ec8c-maas-api-tls\") pod \"maas-api-7686dfb796-b4p2h\" (UID: \"73d90d29-b819-4adc-9d92-71611d77ec8c\") " pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" Jul 09 03:51:46.036228 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:46.036205 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h77gl\" (UniqueName: \"kubernetes.io/projected/c084818f-bc07-4c5e-a79c-f7b44e240330-kube-api-access-h77gl\") pod \"payload-processing-58f86c7798-jbk6d\" (UID: \"c084818f-bc07-4c5e-a79c-f7b44e240330\") " pod="openshift-ingress/payload-processing-58f86c7798-jbk6d" Jul 09 03:51:46.129499 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:46.129402 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-jbk6d" Jul 09 03:51:46.221783 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:46.221749 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" Jul 09 03:51:46.291533 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:46.291466 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-jbk6d"] Jul 09 03:51:46.295249 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:51:46.294688 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc084818f_bc07_4c5e_a79c_f7b44e240330.slice/crio-7e51f9a4d67cd851b29d506201b3be05d2469121005b84b510f452da5574fcbe WatchSource:0}: Error finding container 7e51f9a4d67cd851b29d506201b3be05d2469121005b84b510f452da5574fcbe: Status 404 returned error can't find the container with id 7e51f9a4d67cd851b29d506201b3be05d2469121005b84b510f452da5574fcbe Jul 09 03:51:46.398932 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:46.398132 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h"] Jul 09 03:51:46.408150 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:51:46.408102 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d90d29_b819_4adc_9d92_71611d77ec8c.slice/crio-33f8cc58d19970c209d90ceda151175d939f5691d87d59c38c1cbd48db0169a9 WatchSource:0}: Error finding container 33f8cc58d19970c209d90ceda151175d939f5691d87d59c38c1cbd48db0169a9: Status 404 returned error can't find the container with id 33f8cc58d19970c209d90ceda151175d939f5691d87d59c38c1cbd48db0169a9 Jul 09 03:51:46.695688 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:46.695548 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-jbk6d" event={"ID":"c084818f-bc07-4c5e-a79c-f7b44e240330","Type":"ContainerStarted","Data":"7e51f9a4d67cd851b29d506201b3be05d2469121005b84b510f452da5574fcbe"} Jul 09 03:51:46.699122 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:46.699087 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" event={"ID":"73d90d29-b819-4adc-9d92-71611d77ec8c","Type":"ContainerStarted","Data":"33f8cc58d19970c209d90ceda151175d939f5691d87d59c38c1cbd48db0169a9"} Jul 09 03:51:48.714111 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:48.714076 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-jbk6d" event={"ID":"c084818f-bc07-4c5e-a79c-f7b44e240330","Type":"ContainerStarted","Data":"1b00e3b2b4d488754b1be46060d358b0002e74862bcdcb8c8da7c29d70114f52"} Jul 09 03:51:48.714581 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:48.714190 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-jbk6d" Jul 09 03:51:48.715595 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:48.715571 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" event={"ID":"73d90d29-b819-4adc-9d92-71611d77ec8c","Type":"ContainerStarted","Data":"cb87d0435aa987b79f6696ceb39df4f41802eddf210031e356d6ad9071a6c5fd"} Jul 09 03:51:48.715730 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:48.715711 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" Jul 09 03:51:48.731282 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:48.731234 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-jbk6d" podStartSLOduration=1.7905897149999999 podStartE2EDuration="3.731217503s" totalImagesPullingTime="1.940627788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:51:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:51:48.729139446 +0000 UTC m=+579.360799532" watchObservedRunningTime="2026-07-09 03:51:48.731217503 +0000 UTC m=+579.362877639" Jul 09 03:51:48.744576 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:48.744513 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" podStartSLOduration=1.914934577 podStartE2EDuration="3.744492835s" totalImagesPullingTime="1.829558258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:51:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:51:48.744240555 +0000 UTC m=+579.375900641" watchObservedRunningTime="2026-07-09 03:51:48.744492835 +0000 UTC m=+579.376152921" Jul 09 03:51:53.744337 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:53.744246 2566 generic.go:356] "Generic (PLEG): container finished" podID="4953493d-d4b5-4068-b41c-488d1aec4e79" containerID="afcc21dde5e75d987143f97dcf1416e7058c106ccd3927546fde94ff7bc35fd1" exitCode=0 Jul 09 03:51:53.744337 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:53.744321 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" event={"ID":"4953493d-d4b5-4068-b41c-488d1aec4e79","Type":"ContainerDied","Data":"afcc21dde5e75d987143f97dcf1416e7058c106ccd3927546fde94ff7bc35fd1"} Jul 09 03:51:53.744793 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:53.744755 2566 scope.go:117] "RemoveContainer" containerID="afcc21dde5e75d987143f97dcf1416e7058c106ccd3927546fde94ff7bc35fd1" Jul 09 03:51:53.745731 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:53.745702 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 03:51:53.745833 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:53.745784 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 03:51:53.745894 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:53.745832 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 03:51:54.726153 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:54.726127 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7686dfb796-b4p2h" Jul 09 03:51:54.751437 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:54.751403 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" event={"ID":"4953493d-d4b5-4068-b41c-488d1aec4e79","Type":"ContainerStarted","Data":"f7b42c9e4bc1107799e7c652c4df3cbf4e2b4d9977a439c521d8eef7412a8b3c"} Jul 09 03:51:55.208266 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:55.208229 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:51:55.208545 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:55.208519 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fw4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 09 03:51:55.208595 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:55.208572 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" podUID="4953493d-d4b5-4068-b41c-488d1aec4e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 09 03:51:56.209049 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:56.209011 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fw4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 09 03:51:56.209462 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:56.209075 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" podUID="4953493d-d4b5-4068-b41c-488d1aec4e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 09 03:51:57.208334 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:57.208295 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fw4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 09 03:51:57.208536 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:57.208367 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" podUID="4953493d-d4b5-4068-b41c-488d1aec4e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 09 03:51:58.208915 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:58.208874 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fw4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 09 03:51:58.209414 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:58.208958 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" podUID="4953493d-d4b5-4068-b41c-488d1aec4e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 09 03:51:59.211939 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:59.211908 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:51:59.212313 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:59.212294 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:51:59.213031 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:59.213013 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fw4vn" Jul 09 03:51:59.723397 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:51:59.723370 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-jbk6d" Jul 09 03:52:09.829453 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:09.829421 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 03:52:09.831097 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:09.831078 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 03:52:09.833988 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:09.833961 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 03:52:09.835834 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:09.835810 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 03:52:19.621750 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.621713 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7"] Jul 09 03:52:19.627378 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.627349 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.629995 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.629970 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-27f2j\"" Jul 09 03:52:19.631270 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.631246 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 09 03:52:19.631375 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.631289 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 03:52:19.631450 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.631373 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 03:52:19.634593 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.634574 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7"] Jul 09 03:52:19.648551 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.648477 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26aeea0c-57f2-48c6-98ac-358c102975b1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.648762 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.648621 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26aeea0c-57f2-48c6-98ac-358c102975b1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.648762 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.648707 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26aeea0c-57f2-48c6-98ac-358c102975b1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.648762 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.648755 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvbg7\" (UniqueName: \"kubernetes.io/projected/26aeea0c-57f2-48c6-98ac-358c102975b1-kube-api-access-mvbg7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.648990 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.648806 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26aeea0c-57f2-48c6-98ac-358c102975b1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.648990 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.648839 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26aeea0c-57f2-48c6-98ac-358c102975b1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.750057 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.750018 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26aeea0c-57f2-48c6-98ac-358c102975b1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.750057 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.750061 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26aeea0c-57f2-48c6-98ac-358c102975b1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.750342 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.750106 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvbg7\" (UniqueName: \"kubernetes.io/projected/26aeea0c-57f2-48c6-98ac-358c102975b1-kube-api-access-mvbg7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.750342 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.750152 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26aeea0c-57f2-48c6-98ac-358c102975b1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.750833 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.750440 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26aeea0c-57f2-48c6-98ac-358c102975b1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.750833 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.750496 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26aeea0c-57f2-48c6-98ac-358c102975b1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.750833 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.750523 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26aeea0c-57f2-48c6-98ac-358c102975b1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.750833 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.750572 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26aeea0c-57f2-48c6-98ac-358c102975b1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.750833 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.750768 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26aeea0c-57f2-48c6-98ac-358c102975b1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.752498 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.752473 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26aeea0c-57f2-48c6-98ac-358c102975b1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.753039 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.753011 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26aeea0c-57f2-48c6-98ac-358c102975b1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.759338 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.759305 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvbg7\" (UniqueName: \"kubernetes.io/projected/26aeea0c-57f2-48c6-98ac-358c102975b1-kube-api-access-mvbg7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pfdm7\" (UID: \"26aeea0c-57f2-48c6-98ac-358c102975b1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:19.940362 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:19.940326 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:20.088319 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:20.088291 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7"] Jul 09 03:52:20.090440 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:52:20.090410 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26aeea0c_57f2_48c6_98ac_358c102975b1.slice/crio-d87367773b1cf2b33300bad0356218daacf72a1f8d4ded8cedd7b6cf1c0a1592 WatchSource:0}: Error finding container d87367773b1cf2b33300bad0356218daacf72a1f8d4ded8cedd7b6cf1c0a1592: Status 404 returned error can't find the container with id d87367773b1cf2b33300bad0356218daacf72a1f8d4ded8cedd7b6cf1c0a1592 Jul 09 03:52:20.884836 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:20.884798 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" event={"ID":"26aeea0c-57f2-48c6-98ac-358c102975b1","Type":"ContainerStarted","Data":"d87367773b1cf2b33300bad0356218daacf72a1f8d4ded8cedd7b6cf1c0a1592"} Jul 09 03:52:21.892022 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:21.891981 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" event={"ID":"26aeea0c-57f2-48c6-98ac-358c102975b1","Type":"ContainerStarted","Data":"3eb0de764f67931d99b2d4741f11f774d3e2cbe46e3a6990f049b03f33233915"} Jul 09 03:52:21.909258 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:21.909199 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" podStartSLOduration=1.855319658 podStartE2EDuration="2.909181381s" totalImagesPullingTime="1.053861723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:52:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:52:21.90776721 +0000 UTC m=+612.539427300" watchObservedRunningTime="2026-07-09 03:52:21.909181381 +0000 UTC m=+612.540841466" Jul 09 03:52:29.941251 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:29.941213 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:29.941799 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:29.941283 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:29.953882 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:29.953860 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:30.949149 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:30.949120 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pfdm7" Jul 09 03:52:45.233779 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.233740 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx"] Jul 09 03:52:45.237776 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.237751 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.240257 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.240235 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 09 03:52:45.247007 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.246974 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx"] Jul 09 03:52:45.386246 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.386199 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f86a35-459b-4416-96d7-700ead2442c5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.386246 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.386249 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77f86a35-459b-4416-96d7-700ead2442c5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.386551 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.386289 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvwdk\" (UniqueName: \"kubernetes.io/projected/77f86a35-459b-4416-96d7-700ead2442c5-kube-api-access-bvwdk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.386551 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.386422 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77f86a35-459b-4416-96d7-700ead2442c5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.386551 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.386464 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77f86a35-459b-4416-96d7-700ead2442c5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.386551 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.386488 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f86a35-459b-4416-96d7-700ead2442c5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.487776 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.487638 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77f86a35-459b-4416-96d7-700ead2442c5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.487776 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.487722 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77f86a35-459b-4416-96d7-700ead2442c5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.487776 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.487748 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f86a35-459b-4416-96d7-700ead2442c5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.488086 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.487822 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f86a35-459b-4416-96d7-700ead2442c5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.488086 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.487852 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77f86a35-459b-4416-96d7-700ead2442c5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.488086 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.488000 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77f86a35-459b-4416-96d7-700ead2442c5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.488086 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.488069 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvwdk\" (UniqueName: \"kubernetes.io/projected/77f86a35-459b-4416-96d7-700ead2442c5-kube-api-access-bvwdk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.488279 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.488178 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77f86a35-459b-4416-96d7-700ead2442c5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.488279 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.488212 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f86a35-459b-4416-96d7-700ead2442c5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.490136 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.490107 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77f86a35-459b-4416-96d7-700ead2442c5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.490309 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.490289 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f86a35-459b-4416-96d7-700ead2442c5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.495902 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.495878 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvwdk\" (UniqueName: \"kubernetes.io/projected/77f86a35-459b-4416-96d7-700ead2442c5-kube-api-access-bvwdk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx\" (UID: \"77f86a35-459b-4416-96d7-700ead2442c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.549961 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.549921 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:45.694171 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.694144 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx"] Jul 09 03:52:45.696736 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:52:45.696705 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f86a35_459b_4416_96d7_700ead2442c5.slice/crio-c02dd92d99b8d9fd8aa92cd3b9f01dcf6a36c15bedac18fdb39712182f61dad9 WatchSource:0}: Error finding container c02dd92d99b8d9fd8aa92cd3b9f01dcf6a36c15bedac18fdb39712182f61dad9: Status 404 returned error can't find the container with id c02dd92d99b8d9fd8aa92cd3b9f01dcf6a36c15bedac18fdb39712182f61dad9 Jul 09 03:52:45.698537 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:45.698517 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 03:52:46.018228 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:46.018186 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" event={"ID":"77f86a35-459b-4416-96d7-700ead2442c5","Type":"ContainerStarted","Data":"046f9a9bf1a7ef99ac983042cce0cc3b9f30412b05b0eee1cab2d2dbf255fec1"} Jul 09 03:52:46.018574 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:46.018252 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" event={"ID":"77f86a35-459b-4416-96d7-700ead2442c5","Type":"ContainerStarted","Data":"c02dd92d99b8d9fd8aa92cd3b9f01dcf6a36c15bedac18fdb39712182f61dad9"} Jul 09 03:52:46.035722 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:46.035668 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" podStartSLOduration=0.862829187 podStartE2EDuration="1.035635539s" totalImagesPullingTime="172.806352ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:52:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:52:46.033777356 +0000 UTC m=+636.665437651" watchObservedRunningTime="2026-07-09 03:52:46.035635539 +0000 UTC m=+636.667295624" Jul 09 03:52:55.550506 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:55.550470 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:55.551127 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:55.550543 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:55.563330 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:55.563304 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:52:56.084723 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:52:56.084691 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx" Jul 09 03:54:59.530083 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:54:59.530046 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-869986db6b-ptn66"] Jul 09 03:54:59.530725 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:54:59.530329 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-869986db6b-ptn66" podUID="7c04e044-0ac5-4684-bb47-6f6762adea36" containerName="manager" containerID="cri-o://a83e5aa8df71c75b565fd3251022c070e3dfa5da461c8e4a29a6c8cdd2c8a861" gracePeriod=30 Jul 09 03:54:59.746526 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:54:59.746484 2566 generic.go:356] "Generic (PLEG): container finished" podID="7c04e044-0ac5-4684-bb47-6f6762adea36" containerID="a83e5aa8df71c75b565fd3251022c070e3dfa5da461c8e4a29a6c8cdd2c8a861" exitCode=0 Jul 09 03:54:59.746751 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:54:59.746544 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869986db6b-ptn66" event={"ID":"7c04e044-0ac5-4684-bb47-6f6762adea36","Type":"ContainerDied","Data":"a83e5aa8df71c75b565fd3251022c070e3dfa5da461c8e4a29a6c8cdd2c8a861"} Jul 09 03:54:59.796523 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:54:59.796447 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869986db6b-ptn66" Jul 09 03:54:59.938789 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:54:59.938740 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c04e044-0ac5-4684-bb47-6f6762adea36-webhook-cert\") pod \"7c04e044-0ac5-4684-bb47-6f6762adea36\" (UID: \"7c04e044-0ac5-4684-bb47-6f6762adea36\") " Jul 09 03:54:59.939006 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:54:59.938846 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zt46t\" (UniqueName: \"kubernetes.io/projected/7c04e044-0ac5-4684-bb47-6f6762adea36-kube-api-access-zt46t\") pod \"7c04e044-0ac5-4684-bb47-6f6762adea36\" (UID: \"7c04e044-0ac5-4684-bb47-6f6762adea36\") " Jul 09 03:54:59.940941 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:54:59.940909 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c04e044-0ac5-4684-bb47-6f6762adea36-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7c04e044-0ac5-4684-bb47-6f6762adea36" (UID: "7c04e044-0ac5-4684-bb47-6f6762adea36"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 03:54:59.941055 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:54:59.940996 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c04e044-0ac5-4684-bb47-6f6762adea36-kube-api-access-zt46t" (OuterVolumeSpecName: "kube-api-access-zt46t") pod "7c04e044-0ac5-4684-bb47-6f6762adea36" (UID: "7c04e044-0ac5-4684-bb47-6f6762adea36"). InnerVolumeSpecName "kube-api-access-zt46t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 03:55:00.039920 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:00.039883 2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c04e044-0ac5-4684-bb47-6f6762adea36-webhook-cert\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:55:00.039920 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:00.039914 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt46t\" (UniqueName: \"kubernetes.io/projected/7c04e044-0ac5-4684-bb47-6f6762adea36-kube-api-access-zt46t\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 03:55:00.753130 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:00.753099 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869986db6b-ptn66" Jul 09 03:55:00.753634 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:00.753102 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869986db6b-ptn66" event={"ID":"7c04e044-0ac5-4684-bb47-6f6762adea36","Type":"ContainerDied","Data":"18c45849f1c9850c04fe7bf6c3d272023e40e1148946e421fcd9e76f85804bf1"} Jul 09 03:55:00.753634 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:00.753241 2566 scope.go:117] "RemoveContainer" containerID="a83e5aa8df71c75b565fd3251022c070e3dfa5da461c8e4a29a6c8cdd2c8a861" Jul 09 03:55:00.779466 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:00.779430 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-869986db6b-ptn66"] Jul 09 03:55:00.785080 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:00.785047 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-869986db6b-ptn66"] Jul 09 03:55:01.525414 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.525369 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-869986db6b-kzj9d"] Jul 09 03:55:01.525899 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.525881 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c04e044-0ac5-4684-bb47-6f6762adea36" containerName="manager" Jul 09 03:55:01.526002 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.525902 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c04e044-0ac5-4684-bb47-6f6762adea36" containerName="manager" Jul 09 03:55:01.526056 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.526039 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c04e044-0ac5-4684-bb47-6f6762adea36" containerName="manager" Jul 09 03:55:01.530519 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.530494 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869986db6b-kzj9d" Jul 09 03:55:01.533299 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.533273 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 03:55:01.533299 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.533295 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 03:55:01.533491 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.533321 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qvsk5\"" Jul 09 03:55:01.538162 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.538054 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869986db6b-kzj9d"] Jul 09 03:55:01.657197 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.657158 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcctj\" (UniqueName: \"kubernetes.io/projected/71bc4735-8713-43ac-8d85-4e4cbf21a181-kube-api-access-zcctj\") pod \"maas-controller-869986db6b-kzj9d\" (UID: \"71bc4735-8713-43ac-8d85-4e4cbf21a181\") " pod="opendatahub/maas-controller-869986db6b-kzj9d" Jul 09 03:55:01.657389 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.657227 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71bc4735-8713-43ac-8d85-4e4cbf21a181-webhook-cert\") pod \"maas-controller-869986db6b-kzj9d\" (UID: \"71bc4735-8713-43ac-8d85-4e4cbf21a181\") " pod="opendatahub/maas-controller-869986db6b-kzj9d" Jul 09 03:55:01.758508 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.758468 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71bc4735-8713-43ac-8d85-4e4cbf21a181-webhook-cert\") pod \"maas-controller-869986db6b-kzj9d\" (UID: \"71bc4735-8713-43ac-8d85-4e4cbf21a181\") " pod="opendatahub/maas-controller-869986db6b-kzj9d" Jul 09 03:55:01.759017 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.758590 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcctj\" (UniqueName: \"kubernetes.io/projected/71bc4735-8713-43ac-8d85-4e4cbf21a181-kube-api-access-zcctj\") pod \"maas-controller-869986db6b-kzj9d\" (UID: \"71bc4735-8713-43ac-8d85-4e4cbf21a181\") " pod="opendatahub/maas-controller-869986db6b-kzj9d" Jul 09 03:55:01.761221 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.761199 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71bc4735-8713-43ac-8d85-4e4cbf21a181-webhook-cert\") pod \"maas-controller-869986db6b-kzj9d\" (UID: \"71bc4735-8713-43ac-8d85-4e4cbf21a181\") " pod="opendatahub/maas-controller-869986db6b-kzj9d" Jul 09 03:55:01.767299 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.767276 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcctj\" (UniqueName: \"kubernetes.io/projected/71bc4735-8713-43ac-8d85-4e4cbf21a181-kube-api-access-zcctj\") pod \"maas-controller-869986db6b-kzj9d\" (UID: \"71bc4735-8713-43ac-8d85-4e4cbf21a181\") " pod="opendatahub/maas-controller-869986db6b-kzj9d" Jul 09 03:55:01.843410 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.843312 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869986db6b-kzj9d" Jul 09 03:55:01.867847 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.867812 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c04e044-0ac5-4684-bb47-6f6762adea36" path="/var/lib/kubelet/pods/7c04e044-0ac5-4684-bb47-6f6762adea36/volumes" Jul 09 03:55:01.988845 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:01.988816 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869986db6b-kzj9d"] Jul 09 03:55:01.991518 ip-10-0-137-230 kubenswrapper[2566]: W0709 03:55:01.991487 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71bc4735_8713_43ac_8d85_4e4cbf21a181.slice/crio-ddaa8d87c443a1c9d218f1d42eb1459872d88477bac35d21673ab8d733d05300 WatchSource:0}: Error finding container ddaa8d87c443a1c9d218f1d42eb1459872d88477bac35d21673ab8d733d05300: Status 404 returned error can't find the container with id ddaa8d87c443a1c9d218f1d42eb1459872d88477bac35d21673ab8d733d05300 Jul 09 03:55:02.766983 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:02.766944 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869986db6b-kzj9d" event={"ID":"71bc4735-8713-43ac-8d85-4e4cbf21a181","Type":"ContainerStarted","Data":"80f82901fef0fc59fa60f1f179fafe680844cc9e8d6bbd27d2f73a928a929aae"} Jul 09 03:55:02.767446 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:02.767008 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869986db6b-kzj9d" event={"ID":"71bc4735-8713-43ac-8d85-4e4cbf21a181","Type":"ContainerStarted","Data":"ddaa8d87c443a1c9d218f1d42eb1459872d88477bac35d21673ab8d733d05300"} Jul 09 03:55:02.767446 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:02.767114 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-869986db6b-kzj9d" Jul 09 03:55:02.784287 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:02.784232 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-869986db6b-kzj9d" podStartSLOduration=1.196266067 podStartE2EDuration="1.784213132s" totalImagesPullingTime="587.947065ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:55:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:55:02.782496934 +0000 UTC m=+773.414157020" watchObservedRunningTime="2026-07-09 03:55:02.784213132 +0000 UTC m=+773.415873218" Jul 09 03:55:13.777613 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:55:13.777579 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-869986db6b-kzj9d" Jul 09 03:56:09.910302 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:56:09.910269 2566 scope.go:117] "RemoveContainer" containerID="3faea9725a5b9126aaac905eefde461fa266407ca6f2c565548d2b29af8a58bb" Jul 09 03:57:09.864023 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:57:09.863944 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 03:57:09.864023 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:57:09.863968 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 03:57:09.869202 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:57:09.869182 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 03:57:09.869592 ip-10-0-137-230 kubenswrapper[2566]: I0709 03:57:09.869576 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 04:02:09.898090 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:02:09.898060 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 04:02:09.900483 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:02:09.898383 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 04:02:09.902938 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:02:09.902912 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 04:02:09.903198 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:02:09.903175 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 04:07:09.930001 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:09.929897 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 04:07:09.933454 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:09.930638 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 04:07:09.934382 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:09.934364 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 04:07:09.935178 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:09.935161 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 04:07:12.663112 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.663075 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct"] Jul 09 04:07:12.666231 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.666207 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.668994 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.668967 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1c6bc5d3-openshift-default-dockercfg-fnh58\"" Jul 09 04:07:12.694949 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.694917 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct"] Jul 09 04:07:12.777309 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.777261 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-workload-socket\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.777484 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.777381 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-token\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.777484 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.777418 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-podinfo\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.777484 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.777441 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdbtd\" (UniqueName: \"kubernetes.io/projected/625a40e3-71c6-4603-90f0-26d15e79f95d-kube-api-access-pdbtd\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.777595 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.777547 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-data\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.777595 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.777581 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-envoy\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.777684 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.777663 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-credential-socket\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.777726 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.777690 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-workload-certs\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.777764 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.777728 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/625a40e3-71c6-4603-90f0-26d15e79f95d-istiod-ca-cert\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.879116 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.879066 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-data\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.879316 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.879131 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-envoy\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.879316 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.879189 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-credential-socket\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.879316 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.879214 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-workload-certs\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.879316 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.879261 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/625a40e3-71c6-4603-90f0-26d15e79f95d-istiod-ca-cert\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.879316 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.879306 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-workload-socket\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.879598 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.879369 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-token\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.879598 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.879397 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-podinfo\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.879598 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.879426 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdbtd\" (UniqueName: \"kubernetes.io/projected/625a40e3-71c6-4603-90f0-26d15e79f95d-kube-api-access-pdbtd\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.879598 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.879557 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-data\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.879840 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.879618 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-credential-socket\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.879840 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.879680 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-workload-certs\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.879840 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.879777 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-workload-socket\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.880034 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.880011 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/625a40e3-71c6-4603-90f0-26d15e79f95d-istiod-ca-cert\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.881685 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.881625 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-envoy\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.882007 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.881988 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-podinfo\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.887865 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.887841 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-token\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.887971 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.887944 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdbtd\" (UniqueName: \"kubernetes.io/projected/625a40e3-71c6-4603-90f0-26d15e79f95d-kube-api-access-pdbtd\") pod \"e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:12.978458 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:12.978363 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:13.127638 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:13.127613 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct"] Jul 09 04:07:13.129392 ip-10-0-137-230 kubenswrapper[2566]: W0709 04:07:13.129361 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod625a40e3_71c6_4603_90f0_26d15e79f95d.slice/crio-d6f93ad2d5537d9e7cd77f76eb531da4653f0533b8382361ec52af6c04713da6 WatchSource:0}: Error finding container d6f93ad2d5537d9e7cd77f76eb531da4653f0533b8382361ec52af6c04713da6: Status 404 returned error can't find the container with id d6f93ad2d5537d9e7cd77f76eb531da4653f0533b8382361ec52af6c04713da6 Jul 09 04:07:13.131577 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:13.131531 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:07:13.131683 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:13.131635 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:07:13.131733 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:13.131706 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:07:13.548379 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:13.548339 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" event={"ID":"625a40e3-71c6-4603-90f0-26d15e79f95d","Type":"ContainerStarted","Data":"9fca07dc167b8f4235f75192f52649fd4fc961baac1eac0083e05d79baebc9af"} Jul 09 04:07:13.548572 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:13.548399 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" event={"ID":"625a40e3-71c6-4603-90f0-26d15e79f95d","Type":"ContainerStarted","Data":"d6f93ad2d5537d9e7cd77f76eb531da4653f0533b8382361ec52af6c04713da6"} Jul 09 04:07:13.573480 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:13.573424 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" podStartSLOduration=1.573405246 podStartE2EDuration="1.573405246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:13.571087509 +0000 UTC m=+1504.202747595" watchObservedRunningTime="2026-07-09 04:07:13.573405246 +0000 UTC m=+1504.205065332" Jul 09 04:07:13.978951 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:13.978908 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:14.984400 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:14.984366 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:15.559253 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:15.559214 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:15.560292 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:15.560268 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:19.053129 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:19.053093 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct"] Jul 09 04:07:19.053562 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:19.053277 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" podUID="625a40e3-71c6-4603-90f0-26d15e79f95d" containerName="istio-proxy" containerID="cri-o://9fca07dc167b8f4235f75192f52649fd4fc961baac1eac0083e05d79baebc9af" gracePeriod=30 Jul 09 04:07:24.311163 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.311126 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:24.378805 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.378735 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-workload-socket\") pod \"625a40e3-71c6-4603-90f0-26d15e79f95d\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " Jul 09 04:07:24.378805 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.378778 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-workload-certs\") pod \"625a40e3-71c6-4603-90f0-26d15e79f95d\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " Jul 09 04:07:24.379106 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.378810 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-credential-socket\") pod \"625a40e3-71c6-4603-90f0-26d15e79f95d\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " Jul 09 04:07:24.379106 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.378862 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-envoy\") pod \"625a40e3-71c6-4603-90f0-26d15e79f95d\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " Jul 09 04:07:24.379106 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.378888 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-token\") pod \"625a40e3-71c6-4603-90f0-26d15e79f95d\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " Jul 09 04:07:24.379106 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.378926 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-data\") pod \"625a40e3-71c6-4603-90f0-26d15e79f95d\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " Jul 09 04:07:24.379106 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.379000 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-podinfo\") pod \"625a40e3-71c6-4603-90f0-26d15e79f95d\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " Jul 09 04:07:24.379106 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.379027 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdbtd\" (UniqueName: \"kubernetes.io/projected/625a40e3-71c6-4603-90f0-26d15e79f95d-kube-api-access-pdbtd\") pod \"625a40e3-71c6-4603-90f0-26d15e79f95d\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " Jul 09 04:07:24.379106 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.379051 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/625a40e3-71c6-4603-90f0-26d15e79f95d-istiod-ca-cert\") pod \"625a40e3-71c6-4603-90f0-26d15e79f95d\" (UID: \"625a40e3-71c6-4603-90f0-26d15e79f95d\") " Jul 09 04:07:24.379106 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.379056 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "625a40e3-71c6-4603-90f0-26d15e79f95d" (UID: "625a40e3-71c6-4603-90f0-26d15e79f95d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:24.379106 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.379076 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "625a40e3-71c6-4603-90f0-26d15e79f95d" (UID: "625a40e3-71c6-4603-90f0-26d15e79f95d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:24.379550 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.379184 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "625a40e3-71c6-4603-90f0-26d15e79f95d" (UID: "625a40e3-71c6-4603-90f0-26d15e79f95d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:24.379550 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.379363 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-data" (OuterVolumeSpecName: "istio-data") pod "625a40e3-71c6-4603-90f0-26d15e79f95d" (UID: "625a40e3-71c6-4603-90f0-26d15e79f95d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:24.379550 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.379448 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-workload-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:24.379550 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.379457 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/625a40e3-71c6-4603-90f0-26d15e79f95d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "625a40e3-71c6-4603-90f0-26d15e79f95d" (UID: "625a40e3-71c6-4603-90f0-26d15e79f95d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:07:24.379550 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.379468 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-workload-certs\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:24.379550 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.379482 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-credential-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:24.381406 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.381382 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "625a40e3-71c6-4603-90f0-26d15e79f95d" (UID: "625a40e3-71c6-4603-90f0-26d15e79f95d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:24.381529 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.381466 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/625a40e3-71c6-4603-90f0-26d15e79f95d-kube-api-access-pdbtd" (OuterVolumeSpecName: "kube-api-access-pdbtd") pod "625a40e3-71c6-4603-90f0-26d15e79f95d" (UID: "625a40e3-71c6-4603-90f0-26d15e79f95d"). InnerVolumeSpecName "kube-api-access-pdbtd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:07:24.381694 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.381672 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "625a40e3-71c6-4603-90f0-26d15e79f95d" (UID: "625a40e3-71c6-4603-90f0-26d15e79f95d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:07:24.381761 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.381707 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-token" (OuterVolumeSpecName: "istio-token") pod "625a40e3-71c6-4603-90f0-26d15e79f95d" (UID: "625a40e3-71c6-4603-90f0-26d15e79f95d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:07:24.480423 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.480363 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-data\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:24.480423 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.480412 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-podinfo\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:24.480423 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.480422 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdbtd\" (UniqueName: \"kubernetes.io/projected/625a40e3-71c6-4603-90f0-26d15e79f95d-kube-api-access-pdbtd\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:24.480423 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.480434 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/625a40e3-71c6-4603-90f0-26d15e79f95d-istiod-ca-cert\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:24.480423 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.480443 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-envoy\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:24.480793 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.480451 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/625a40e3-71c6-4603-90f0-26d15e79f95d-istio-token\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:24.610808 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.610766 2566 generic.go:356] "Generic (PLEG): container finished" podID="625a40e3-71c6-4603-90f0-26d15e79f95d" containerID="9fca07dc167b8f4235f75192f52649fd4fc961baac1eac0083e05d79baebc9af" exitCode=0 Jul 09 04:07:24.610997 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.610825 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" Jul 09 04:07:24.610997 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.610853 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" event={"ID":"625a40e3-71c6-4603-90f0-26d15e79f95d","Type":"ContainerDied","Data":"9fca07dc167b8f4235f75192f52649fd4fc961baac1eac0083e05d79baebc9af"} Jul 09 04:07:24.610997 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.610916 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct" event={"ID":"625a40e3-71c6-4603-90f0-26d15e79f95d","Type":"ContainerDied","Data":"d6f93ad2d5537d9e7cd77f76eb531da4653f0533b8382361ec52af6c04713da6"} Jul 09 04:07:24.610997 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.610939 2566 scope.go:117] "RemoveContainer" containerID="9fca07dc167b8f4235f75192f52649fd4fc961baac1eac0083e05d79baebc9af" Jul 09 04:07:24.620981 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.620960 2566 scope.go:117] "RemoveContainer" containerID="9fca07dc167b8f4235f75192f52649fd4fc961baac1eac0083e05d79baebc9af" Jul 09 04:07:24.621284 ip-10-0-137-230 kubenswrapper[2566]: E0709 04:07:24.621260 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fca07dc167b8f4235f75192f52649fd4fc961baac1eac0083e05d79baebc9af\": container with ID starting with 9fca07dc167b8f4235f75192f52649fd4fc961baac1eac0083e05d79baebc9af not found: ID does not exist" containerID="9fca07dc167b8f4235f75192f52649fd4fc961baac1eac0083e05d79baebc9af" Jul 09 04:07:24.621385 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.621290 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fca07dc167b8f4235f75192f52649fd4fc961baac1eac0083e05d79baebc9af"} err="failed to get container status \"9fca07dc167b8f4235f75192f52649fd4fc961baac1eac0083e05d79baebc9af\": rpc error: code = NotFound desc = could not find container \"9fca07dc167b8f4235f75192f52649fd4fc961baac1eac0083e05d79baebc9af\": container with ID starting with 9fca07dc167b8f4235f75192f52649fd4fc961baac1eac0083e05d79baebc9af not found: ID does not exist" Jul 09 04:07:24.632802 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.632737 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct"] Jul 09 04:07:24.636561 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:24.636536 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1c6bc5d3-openshift-default-7d749c8cc7-m87ct"] Jul 09 04:07:25.234709 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.234679 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk"] Jul 09 04:07:25.235148 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.235130 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="625a40e3-71c6-4603-90f0-26d15e79f95d" containerName="istio-proxy" Jul 09 04:07:25.235231 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.235151 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="625a40e3-71c6-4603-90f0-26d15e79f95d" containerName="istio-proxy" Jul 09 04:07:25.235294 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.235286 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="625a40e3-71c6-4603-90f0-26d15e79f95d" containerName="istio-proxy" Jul 09 04:07:25.239767 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.239747 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.242932 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.242781 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5294925d-openshift-default-dockercfg-2jffz\"" Jul 09 04:07:25.250470 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.250447 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk"] Jul 09 04:07:25.285288 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.285250 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istiod-ca-cert\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.285467 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.285304 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-workload-certs\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.285467 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.285338 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-data\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.285467 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.285357 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-token\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.285467 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.285380 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-workload-socket\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.285643 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.285488 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-podinfo\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.285643 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.285529 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr26c\" (UniqueName: \"kubernetes.io/projected/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-kube-api-access-gr26c\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.285643 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.285550 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-credential-socket\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.285643 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.285582 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-envoy\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.386278 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.386238 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-podinfo\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.386278 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.386286 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr26c\" (UniqueName: \"kubernetes.io/projected/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-kube-api-access-gr26c\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.386863 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.386318 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-credential-socket\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.386863 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.386335 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-envoy\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.386863 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.386365 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istiod-ca-cert\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.386863 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.386503 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-workload-certs\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.386863 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.386546 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-data\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.386863 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.386565 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-token\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.386863 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.386640 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-workload-socket\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.387159 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.386895 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-credential-socket\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.387159 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.386992 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-data\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.387159 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.387002 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-workload-certs\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.387159 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.387070 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-workload-socket\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.387292 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.387251 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istiod-ca-cert\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.388835 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.388814 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-podinfo\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.389056 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.389038 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-envoy\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.395257 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.395237 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr26c\" (UniqueName: \"kubernetes.io/projected/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-kube-api-access-gr26c\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.395387 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.395368 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-token\") pod \"e2e-ait-5294925d-openshift-default-9798d7f95-f9szk\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.553829 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.553727 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:25.710596 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.710547 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk"] Jul 09 04:07:25.713086 ip-10-0-137-230 kubenswrapper[2566]: W0709 04:07:25.713058 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78d99a67_e9c8_4a61_ab8e_6bfb144ab121.slice/crio-f8e80d46e061041ca6877b2135d8116b06ee440e0cae51fb938cb3831ea61201 WatchSource:0}: Error finding container f8e80d46e061041ca6877b2135d8116b06ee440e0cae51fb938cb3831ea61201: Status 404 returned error can't find the container with id f8e80d46e061041ca6877b2135d8116b06ee440e0cae51fb938cb3831ea61201 Jul 09 04:07:25.715319 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.715283 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:07:25.715413 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.715350 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:07:25.715413 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.715381 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:07:25.866597 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:25.866510 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="625a40e3-71c6-4603-90f0-26d15e79f95d" path="/var/lib/kubelet/pods/625a40e3-71c6-4603-90f0-26d15e79f95d/volumes" Jul 09 04:07:26.627690 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:26.627640 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" event={"ID":"78d99a67-e9c8-4a61-ab8e-6bfb144ab121","Type":"ContainerStarted","Data":"8444c85e7ac1fb721e84f705b4b529769ce3a691df424e3d3c487b3bb133c302"} Jul 09 04:07:26.628131 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:26.627711 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" event={"ID":"78d99a67-e9c8-4a61-ab8e-6bfb144ab121","Type":"ContainerStarted","Data":"f8e80d46e061041ca6877b2135d8116b06ee440e0cae51fb938cb3831ea61201"} Jul 09 04:07:26.659583 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:26.659529 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" podStartSLOduration=1.659512396 podStartE2EDuration="1.659512396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:26.657792483 +0000 UTC m=+1517.289452570" watchObservedRunningTime="2026-07-09 04:07:26.659512396 +0000 UTC m=+1517.291172482" Jul 09 04:07:27.555046 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:27.555006 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:27.560277 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:27.560248 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:27.632880 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:27.632844 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:27.633975 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:27.633958 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:37.497206 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:37.497158 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk"] Jul 09 04:07:37.497717 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:37.497481 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" podUID="78d99a67-e9c8-4a61-ab8e-6bfb144ab121" containerName="istio-proxy" containerID="cri-o://8444c85e7ac1fb721e84f705b4b529769ce3a691df424e3d3c487b3bb133c302" gracePeriod=30 Jul 09 04:07:38.285993 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.285961 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:38.304933 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.304815 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gr26c\" (UniqueName: \"kubernetes.io/projected/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-kube-api-access-gr26c\") pod \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " Jul 09 04:07:38.304933 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.304863 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-credential-socket\") pod \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " Jul 09 04:07:38.304933 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.304905 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-workload-socket\") pod \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " Jul 09 04:07:38.304933 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.304932 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-workload-certs\") pod \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " Jul 09 04:07:38.305258 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.304996 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-podinfo\") pod \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " Jul 09 04:07:38.305258 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.305025 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-envoy\") pod \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " Jul 09 04:07:38.305258 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.305122 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istiod-ca-cert\") pod \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " Jul 09 04:07:38.305258 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.305147 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-data\") pod \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " Jul 09 04:07:38.305258 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.305172 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-token\") pod \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\" (UID: \"78d99a67-e9c8-4a61-ab8e-6bfb144ab121\") " Jul 09 04:07:38.305258 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.305170 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "78d99a67-e9c8-4a61-ab8e-6bfb144ab121" (UID: "78d99a67-e9c8-4a61-ab8e-6bfb144ab121"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:38.305542 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.305498 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-credential-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:38.305590 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.305543 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "78d99a67-e9c8-4a61-ab8e-6bfb144ab121" (UID: "78d99a67-e9c8-4a61-ab8e-6bfb144ab121"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:07:38.306016 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.305985 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "78d99a67-e9c8-4a61-ab8e-6bfb144ab121" (UID: "78d99a67-e9c8-4a61-ab8e-6bfb144ab121"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:38.306222 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.306181 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "78d99a67-e9c8-4a61-ab8e-6bfb144ab121" (UID: "78d99a67-e9c8-4a61-ab8e-6bfb144ab121"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:38.306804 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.306762 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-data" (OuterVolumeSpecName: "istio-data") pod "78d99a67-e9c8-4a61-ab8e-6bfb144ab121" (UID: "78d99a67-e9c8-4a61-ab8e-6bfb144ab121"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:38.307905 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.307859 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-kube-api-access-gr26c" (OuterVolumeSpecName: "kube-api-access-gr26c") pod "78d99a67-e9c8-4a61-ab8e-6bfb144ab121" (UID: "78d99a67-e9c8-4a61-ab8e-6bfb144ab121"). InnerVolumeSpecName "kube-api-access-gr26c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:07:38.308108 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.308079 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "78d99a67-e9c8-4a61-ab8e-6bfb144ab121" (UID: "78d99a67-e9c8-4a61-ab8e-6bfb144ab121"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:07:38.308212 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.308176 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-token" (OuterVolumeSpecName: "istio-token") pod "78d99a67-e9c8-4a61-ab8e-6bfb144ab121" (UID: "78d99a67-e9c8-4a61-ab8e-6bfb144ab121"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:07:38.308895 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.308870 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "78d99a67-e9c8-4a61-ab8e-6bfb144ab121" (UID: "78d99a67-e9c8-4a61-ab8e-6bfb144ab121"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:38.406663 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.406619 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istiod-ca-cert\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:38.406858 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.406685 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-data\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:38.406858 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.406694 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-token\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:38.406858 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.406702 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gr26c\" (UniqueName: \"kubernetes.io/projected/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-kube-api-access-gr26c\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:38.406858 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.406713 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-workload-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:38.406858 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.406722 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-workload-certs\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:38.406858 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.406730 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-podinfo\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:38.406858 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.406738 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78d99a67-e9c8-4a61-ab8e-6bfb144ab121-istio-envoy\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:38.690733 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.690699 2566 generic.go:356] "Generic (PLEG): container finished" podID="78d99a67-e9c8-4a61-ab8e-6bfb144ab121" containerID="8444c85e7ac1fb721e84f705b4b529769ce3a691df424e3d3c487b3bb133c302" exitCode=0 Jul 09 04:07:38.691232 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.690762 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" Jul 09 04:07:38.691232 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.690790 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" event={"ID":"78d99a67-e9c8-4a61-ab8e-6bfb144ab121","Type":"ContainerDied","Data":"8444c85e7ac1fb721e84f705b4b529769ce3a691df424e3d3c487b3bb133c302"} Jul 09 04:07:38.691232 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.690860 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk" event={"ID":"78d99a67-e9c8-4a61-ab8e-6bfb144ab121","Type":"ContainerDied","Data":"f8e80d46e061041ca6877b2135d8116b06ee440e0cae51fb938cb3831ea61201"} Jul 09 04:07:38.691232 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.690897 2566 scope.go:117] "RemoveContainer" containerID="8444c85e7ac1fb721e84f705b4b529769ce3a691df424e3d3c487b3bb133c302" Jul 09 04:07:38.700768 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.700746 2566 scope.go:117] "RemoveContainer" containerID="8444c85e7ac1fb721e84f705b4b529769ce3a691df424e3d3c487b3bb133c302" Jul 09 04:07:38.701067 ip-10-0-137-230 kubenswrapper[2566]: E0709 04:07:38.701045 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8444c85e7ac1fb721e84f705b4b529769ce3a691df424e3d3c487b3bb133c302\": container with ID starting with 8444c85e7ac1fb721e84f705b4b529769ce3a691df424e3d3c487b3bb133c302 not found: ID does not exist" containerID="8444c85e7ac1fb721e84f705b4b529769ce3a691df424e3d3c487b3bb133c302" Jul 09 04:07:38.701170 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.701075 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8444c85e7ac1fb721e84f705b4b529769ce3a691df424e3d3c487b3bb133c302"} err="failed to get container status \"8444c85e7ac1fb721e84f705b4b529769ce3a691df424e3d3c487b3bb133c302\": rpc error: code = NotFound desc = could not find container \"8444c85e7ac1fb721e84f705b4b529769ce3a691df424e3d3c487b3bb133c302\": container with ID starting with 8444c85e7ac1fb721e84f705b4b529769ce3a691df424e3d3c487b3bb133c302 not found: ID does not exist" Jul 09 04:07:38.712262 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.712235 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk"] Jul 09 04:07:38.715564 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:38.715539 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5294925d-openshift-default-9798d7f95-f9szk"] Jul 09 04:07:39.866534 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:39.866497 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78d99a67-e9c8-4a61-ab8e-6bfb144ab121" path="/var/lib/kubelet/pods/78d99a67-e9c8-4a61-ab8e-6bfb144ab121/volumes" Jul 09 04:07:43.628433 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.628401 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d"] Jul 09 04:07:43.628963 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.628878 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78d99a67-e9c8-4a61-ab8e-6bfb144ab121" containerName="istio-proxy" Jul 09 04:07:43.628963 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.628891 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="78d99a67-e9c8-4a61-ab8e-6bfb144ab121" containerName="istio-proxy" Jul 09 04:07:43.629081 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.628974 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="78d99a67-e9c8-4a61-ab8e-6bfb144ab121" containerName="istio-proxy" Jul 09 04:07:43.631878 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.631857 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.635515 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.635497 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-0a452316-openshift-default-dockercfg-lzj2d\"" Jul 09 04:07:43.659409 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.659380 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d"] Jul 09 04:07:43.754304 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.754251 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-workload-socket\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.754733 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.754313 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-credential-socket\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.754733 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.754337 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-data\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.754733 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.754355 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istiod-ca-cert\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.754733 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.754475 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l9vn\" (UniqueName: \"kubernetes.io/projected/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-kube-api-access-2l9vn\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.754733 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.754542 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-envoy\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.754733 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.754598 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-workload-certs\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.754733 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.754682 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-podinfo\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.755068 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.754749 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-token\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.855633 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.855581 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-envoy\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.855633 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.855639 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-workload-certs\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.855930 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.855761 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-podinfo\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.855930 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.855780 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-token\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.855930 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.855855 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-workload-socket\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.855930 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.855876 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-credential-socket\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.855930 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.855892 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-data\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.855930 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.855910 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istiod-ca-cert\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.856242 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.855950 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l9vn\" (UniqueName: \"kubernetes.io/projected/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-kube-api-access-2l9vn\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.856242 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.856032 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-workload-certs\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.856242 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.856191 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-workload-socket\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.856485 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.856460 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-credential-socket\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.856545 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.856482 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-data\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.856693 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.856675 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istiod-ca-cert\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.858127 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.858104 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-envoy\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.858240 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.858216 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-podinfo\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.864582 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.864554 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-token\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.864841 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.864820 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l9vn\" (UniqueName: \"kubernetes.io/projected/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-kube-api-access-2l9vn\") pod \"e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:43.945527 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:43.945488 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:44.086459 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:44.086436 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d"] Jul 09 04:07:44.089514 ip-10-0-137-230 kubenswrapper[2566]: W0709 04:07:44.089487 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda691a385_2a74_4f63_8a8a_5b8f3efc2a6f.slice/crio-7fe81b93d5a9ffd231e7f2e8c1e355ea47f2752205f6f459fd4d54bf834316bf WatchSource:0}: Error finding container 7fe81b93d5a9ffd231e7f2e8c1e355ea47f2752205f6f459fd4d54bf834316bf: Status 404 returned error can't find the container with id 7fe81b93d5a9ffd231e7f2e8c1e355ea47f2752205f6f459fd4d54bf834316bf Jul 09 04:07:44.091578 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:44.091546 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:07:44.091674 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:44.091618 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:07:44.091674 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:44.091664 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:07:44.727890 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:44.727850 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" event={"ID":"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f","Type":"ContainerStarted","Data":"2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50"} Jul 09 04:07:44.728324 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:44.727937 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" event={"ID":"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f","Type":"ContainerStarted","Data":"7fe81b93d5a9ffd231e7f2e8c1e355ea47f2752205f6f459fd4d54bf834316bf"} Jul 09 04:07:44.748825 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:44.748776 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" podStartSLOduration=1.748760054 podStartE2EDuration="1.748760054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:44.747347066 +0000 UTC m=+1535.379007153" watchObservedRunningTime="2026-07-09 04:07:44.748760054 +0000 UTC m=+1535.380420138" Jul 09 04:07:44.946340 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:44.946312 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:45.951371 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:45.951342 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:46.739097 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:46.739068 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:46.740211 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:46.740189 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:49.391616 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:49.391570 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d"] Jul 09 04:07:49.753874 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:49.753833 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" podUID="a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" containerName="istio-proxy" containerID="cri-o://2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50" gracePeriod=30 Jul 09 04:07:54.808236 ip-10-0-137-230 kubenswrapper[2566]: E0709 04:07:54.808195 2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda691a385_2a74_4f63_8a8a_5b8f3efc2a6f.slice/crio-2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda691a385_2a74_4f63_8a8a_5b8f3efc2a6f.slice/crio-conmon-2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50.scope\": RecentStats: unable to find data in memory cache]" Jul 09 04:07:55.016396 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.016364 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:55.162116 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162088 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istiod-ca-cert\") pod \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " Jul 09 04:07:55.162410 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162139 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-credential-socket\") pod \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " Jul 09 04:07:55.162410 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162163 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-data\") pod \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " Jul 09 04:07:55.162410 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162190 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-podinfo\") pod \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " Jul 09 04:07:55.162410 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162222 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l9vn\" (UniqueName: \"kubernetes.io/projected/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-kube-api-access-2l9vn\") pod \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " Jul 09 04:07:55.162410 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162252 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-envoy\") pod \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " Jul 09 04:07:55.162410 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162286 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-token\") pod \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " Jul 09 04:07:55.162410 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162315 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-workload-socket\") pod \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " Jul 09 04:07:55.162410 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162368 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-workload-certs\") pod \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\" (UID: \"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f\") " Jul 09 04:07:55.162873 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162429 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" (UID: "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:55.162873 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162488 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-data" (OuterVolumeSpecName: "istio-data") pod "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" (UID: "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:55.162873 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162574 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" (UID: "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:07:55.162873 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162693 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" (UID: "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:55.162873 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162710 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" (UID: "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:55.162873 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162821 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-workload-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:55.162873 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162844 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-workload-certs\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:55.162873 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162858 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istiod-ca-cert\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:55.162873 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162871 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-credential-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:55.163330 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.162886 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-data\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:55.164602 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.164570 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-kube-api-access-2l9vn" (OuterVolumeSpecName: "kube-api-access-2l9vn") pod "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" (UID: "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f"). InnerVolumeSpecName "kube-api-access-2l9vn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:07:55.164739 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.164698 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-token" (OuterVolumeSpecName: "istio-token") pod "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" (UID: "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:07:55.164739 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.164705 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" (UID: "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:07:55.164830 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.164739 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" (UID: "a691a385-2a74-4f63-8a8a-5b8f3efc2a6f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:55.264004 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.263966 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-token\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:55.264004 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.263998 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-podinfo\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:55.264004 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.264008 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l9vn\" (UniqueName: \"kubernetes.io/projected/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-kube-api-access-2l9vn\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:55.264241 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.264019 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f-istio-envoy\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:07:55.785180 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.785142 2566 generic.go:356] "Generic (PLEG): container finished" podID="a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" containerID="2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50" exitCode=0 Jul 09 04:07:55.785384 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.785191 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" Jul 09 04:07:55.785384 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.785219 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" event={"ID":"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f","Type":"ContainerDied","Data":"2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50"} Jul 09 04:07:55.785384 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.785271 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d" event={"ID":"a691a385-2a74-4f63-8a8a-5b8f3efc2a6f","Type":"ContainerDied","Data":"7fe81b93d5a9ffd231e7f2e8c1e355ea47f2752205f6f459fd4d54bf834316bf"} Jul 09 04:07:55.785384 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.785293 2566 scope.go:117] "RemoveContainer" containerID="2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50" Jul 09 04:07:55.795281 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.795264 2566 scope.go:117] "RemoveContainer" containerID="2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50" Jul 09 04:07:55.795567 ip-10-0-137-230 kubenswrapper[2566]: E0709 04:07:55.795549 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50\": container with ID starting with 2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50 not found: ID does not exist" containerID="2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50" Jul 09 04:07:55.795632 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.795577 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50"} err="failed to get container status \"2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50\": rpc error: code = NotFound desc = could not find container \"2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50\": container with ID starting with 2b857eb6df39e1a914adb7ce0e577624d25039e185a20e354508a6314663fa50 not found: ID does not exist" Jul 09 04:07:55.815799 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.815757 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d"] Jul 09 04:07:55.832176 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.832143 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-0a452316-openshift-default-5dddddbcf-6dz2d"] Jul 09 04:07:55.865216 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:07:55.865188 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" path="/var/lib/kubelet/pods/a691a385-2a74-4f63-8a8a-5b8f3efc2a6f/volumes" Jul 09 04:09:29.302575 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.302479 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr"] Jul 09 04:09:29.303249 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.303227 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" containerName="istio-proxy" Jul 09 04:09:29.303349 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.303252 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" containerName="istio-proxy" Jul 09 04:09:29.303349 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.303355 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a691a385-2a74-4f63-8a8a-5b8f3efc2a6f" containerName="istio-proxy" Jul 09 04:09:29.306529 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.306511 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.309184 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.309165 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-99003d6d-openshift-default-dockercfg-qgntq\"" Jul 09 04:09:29.325436 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.325402 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr"] Jul 09 04:09:29.447916 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.447873 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-credential-socket\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.448133 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.447935 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-podinfo\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.448133 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.447957 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istiod-ca-cert\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.448133 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.448048 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-data\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.448133 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.448085 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jg9h\" (UniqueName: \"kubernetes.io/projected/ab71f887-ef79-410a-92fc-5675c6ebc3e1-kube-api-access-7jg9h\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.448315 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.448185 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-workload-socket\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.448315 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.448224 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-workload-certs\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.448315 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.448267 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-envoy\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.448315 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.448298 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-token\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.549235 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.549191 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-envoy\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.549235 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.549243 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-token\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.549523 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.549281 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-credential-socket\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.549523 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.549342 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-podinfo\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.549523 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.549367 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istiod-ca-cert\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.549523 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.549411 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-data\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.549523 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.549438 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jg9h\" (UniqueName: \"kubernetes.io/projected/ab71f887-ef79-410a-92fc-5675c6ebc3e1-kube-api-access-7jg9h\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.549523 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.549501 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-workload-socket\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.549894 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.549530 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-workload-certs\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.549894 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.549811 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-credential-socket\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.549999 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.549906 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-workload-certs\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.550132 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.550089 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-workload-socket\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.550204 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.550141 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-data\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.550204 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.550190 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istiod-ca-cert\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.551819 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.551795 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-envoy\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.551926 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.551881 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-podinfo\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.557508 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.557439 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-token\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.557722 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.557700 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jg9h\" (UniqueName: \"kubernetes.io/projected/ab71f887-ef79-410a-92fc-5675c6ebc3e1-kube-api-access-7jg9h\") pod \"e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.616790 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.616749 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:29.794741 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.794712 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr"] Jul 09 04:09:29.795944 ip-10-0-137-230 kubenswrapper[2566]: W0709 04:09:29.795916 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab71f887_ef79_410a_92fc_5675c6ebc3e1.slice/crio-b3f20d0419cb06b7f9f729ed8823723feb66f19488eb62e143fef12cdecd1936 WatchSource:0}: Error finding container b3f20d0419cb06b7f9f729ed8823723feb66f19488eb62e143fef12cdecd1936: Status 404 returned error can't find the container with id b3f20d0419cb06b7f9f729ed8823723feb66f19488eb62e143fef12cdecd1936 Jul 09 04:09:29.798362 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.798316 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:09:29.798458 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.798418 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:09:29.798500 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:29.798465 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:09:30.268711 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:30.268673 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" event={"ID":"ab71f887-ef79-410a-92fc-5675c6ebc3e1","Type":"ContainerStarted","Data":"29958227f8f1471611472bf8c1af20409300907e1ab575c86bd994c84a8ea3cb"} Jul 09 04:09:30.268898 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:30.268734 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" event={"ID":"ab71f887-ef79-410a-92fc-5675c6ebc3e1","Type":"ContainerStarted","Data":"b3f20d0419cb06b7f9f729ed8823723feb66f19488eb62e143fef12cdecd1936"} Jul 09 04:09:30.288854 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:30.288806 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" podStartSLOduration=1.288785823 podStartE2EDuration="1.288785823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:09:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:09:30.287376888 +0000 UTC m=+1640.919036974" watchObservedRunningTime="2026-07-09 04:09:30.288785823 +0000 UTC m=+1640.920445911" Jul 09 04:09:30.617561 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:30.617467 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:30.622631 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:30.622606 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:31.273717 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:31.273684 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:31.274844 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:31.274822 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:47.067032 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:47.066281 2566 patch_prober.go:28] interesting pod/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:41626->10.132.0.54:15021: read: connection reset by peer" start-of-body= Jul 09 04:09:47.067032 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:47.066380 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" podUID="ab71f887-ef79-410a-92fc-5675c6ebc3e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:41626->10.132.0.54:15021: read: connection reset by peer" Jul 09 04:09:47.361068 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:47.360968 2566 generic.go:356] "Generic (PLEG): container finished" podID="ab71f887-ef79-410a-92fc-5675c6ebc3e1" containerID="29958227f8f1471611472bf8c1af20409300907e1ab575c86bd994c84a8ea3cb" exitCode=0 Jul 09 04:09:47.361068 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:47.361042 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" event={"ID":"ab71f887-ef79-410a-92fc-5675c6ebc3e1","Type":"ContainerDied","Data":"29958227f8f1471611472bf8c1af20409300907e1ab575c86bd994c84a8ea3cb"} Jul 09 04:09:47.361461 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:47.361447 2566 scope.go:117] "RemoveContainer" containerID="29958227f8f1471611472bf8c1af20409300907e1ab575c86bd994c84a8ea3cb" Jul 09 04:09:47.362527 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:47.362493 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:09:47.362614 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:47.362596 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:09:47.362689 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:47.362641 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:09:48.368936 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:48.368905 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" event={"ID":"ab71f887-ef79-410a-92fc-5675c6ebc3e1","Type":"ContainerStarted","Data":"f423239c50b8ca8910cec13225013b20a8b51b48f6a2579e8dceb8a0fc93cffa"} Jul 09 04:09:48.617435 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:48.617400 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:48.617851 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:48.617826 2566 patch_prober.go:28] interesting pod/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 09 04:09:48.617929 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:48.617893 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" podUID="ab71f887-ef79-410a-92fc-5675c6ebc3e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 09 04:09:49.617788 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:49.617755 2566 patch_prober.go:28] interesting pod/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 09 04:09:49.618207 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:49.617824 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" podUID="ab71f887-ef79-410a-92fc-5675c6ebc3e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 09 04:09:50.327464 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:50.327422 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr"] Jul 09 04:09:50.401988 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:50.401941 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" podUID="ab71f887-ef79-410a-92fc-5675c6ebc3e1" containerName="istio-proxy" containerID="cri-o://f423239c50b8ca8910cec13225013b20a8b51b48f6a2579e8dceb8a0fc93cffa" gracePeriod=30 Jul 09 04:09:57.442967 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.442932 2566 generic.go:356] "Generic (PLEG): container finished" podID="ab71f887-ef79-410a-92fc-5675c6ebc3e1" containerID="f423239c50b8ca8910cec13225013b20a8b51b48f6a2579e8dceb8a0fc93cffa" exitCode=0 Jul 09 04:09:57.443415 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.443010 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" event={"ID":"ab71f887-ef79-410a-92fc-5675c6ebc3e1","Type":"ContainerDied","Data":"f423239c50b8ca8910cec13225013b20a8b51b48f6a2579e8dceb8a0fc93cffa"} Jul 09 04:09:57.443415 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.443081 2566 scope.go:117] "RemoveContainer" containerID="29958227f8f1471611472bf8c1af20409300907e1ab575c86bd994c84a8ea3cb" Jul 09 04:09:57.481489 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.481456 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:57.531211 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.531106 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-podinfo\") pod \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " Jul 09 04:09:57.531211 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.531154 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-workload-certs\") pod \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " Jul 09 04:09:57.531211 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.531186 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-workload-socket\") pod \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " Jul 09 04:09:57.531211 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.531207 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-token\") pod \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " Jul 09 04:09:57.531577 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.531234 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jg9h\" (UniqueName: \"kubernetes.io/projected/ab71f887-ef79-410a-92fc-5675c6ebc3e1-kube-api-access-7jg9h\") pod \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " Jul 09 04:09:57.531577 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.531261 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-credential-socket\") pod \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " Jul 09 04:09:57.531577 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.531308 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-envoy\") pod \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " Jul 09 04:09:57.531577 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.531410 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-data\") pod \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " Jul 09 04:09:57.531577 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.531475 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istiod-ca-cert\") pod \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\" (UID: \"ab71f887-ef79-410a-92fc-5675c6ebc3e1\") " Jul 09 04:09:57.531577 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.531554 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab71f887-ef79-410a-92fc-5675c6ebc3e1" (UID: "ab71f887-ef79-410a-92fc-5675c6ebc3e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:09:57.531957 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.531567 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab71f887-ef79-410a-92fc-5675c6ebc3e1" (UID: "ab71f887-ef79-410a-92fc-5675c6ebc3e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:09:57.531957 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.531900 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-workload-certs\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:09:57.531957 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.531925 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-workload-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:09:57.532126 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.531952 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab71f887-ef79-410a-92fc-5675c6ebc3e1" (UID: "ab71f887-ef79-410a-92fc-5675c6ebc3e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:09:57.532710 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.532682 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab71f887-ef79-410a-92fc-5675c6ebc3e1" (UID: "ab71f887-ef79-410a-92fc-5675c6ebc3e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:09:57.532836 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.532779 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab71f887-ef79-410a-92fc-5675c6ebc3e1" (UID: "ab71f887-ef79-410a-92fc-5675c6ebc3e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:09:57.533729 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.533698 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab71f887-ef79-410a-92fc-5675c6ebc3e1" (UID: "ab71f887-ef79-410a-92fc-5675c6ebc3e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:09:57.533729 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.533700 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab71f887-ef79-410a-92fc-5675c6ebc3e1-kube-api-access-7jg9h" (OuterVolumeSpecName: "kube-api-access-7jg9h") pod "ab71f887-ef79-410a-92fc-5675c6ebc3e1" (UID: "ab71f887-ef79-410a-92fc-5675c6ebc3e1"). InnerVolumeSpecName "kube-api-access-7jg9h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:09:57.533879 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.533809 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab71f887-ef79-410a-92fc-5675c6ebc3e1" (UID: "ab71f887-ef79-410a-92fc-5675c6ebc3e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:09:57.533879 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.533853 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab71f887-ef79-410a-92fc-5675c6ebc3e1" (UID: "ab71f887-ef79-410a-92fc-5675c6ebc3e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:09:57.633310 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.633261 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-token\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:09:57.633310 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.633302 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jg9h\" (UniqueName: \"kubernetes.io/projected/ab71f887-ef79-410a-92fc-5675c6ebc3e1-kube-api-access-7jg9h\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:09:57.633310 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.633314 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-credential-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:09:57.633310 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.633323 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-envoy\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:09:57.633607 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.633333 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-data\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:09:57.633607 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.633341 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istiod-ca-cert\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:09:57.633607 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:57.633349 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab71f887-ef79-410a-92fc-5675c6ebc3e1-istio-podinfo\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:09:58.449420 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:58.449391 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" Jul 09 04:09:58.449861 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:58.449378 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr" event={"ID":"ab71f887-ef79-410a-92fc-5675c6ebc3e1","Type":"ContainerDied","Data":"b3f20d0419cb06b7f9f729ed8823723feb66f19488eb62e143fef12cdecd1936"} Jul 09 04:09:58.449861 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:58.449522 2566 scope.go:117] "RemoveContainer" containerID="f423239c50b8ca8910cec13225013b20a8b51b48f6a2579e8dceb8a0fc93cffa" Jul 09 04:09:58.467464 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:58.467428 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr"] Jul 09 04:09:58.473500 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:58.473474 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-99003d6d-openshift-default-7b69b4556c-689sr"] Jul 09 04:09:59.865143 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:09:59.865110 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab71f887-ef79-410a-92fc-5675c6ebc3e1" path="/var/lib/kubelet/pods/ab71f887-ef79-410a-92fc-5675c6ebc3e1/volumes" Jul 09 04:10:09.542154 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.542116 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh"] Jul 09 04:10:09.542837 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.542815 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab71f887-ef79-410a-92fc-5675c6ebc3e1" containerName="istio-proxy" Jul 09 04:10:09.542915 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.542840 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab71f887-ef79-410a-92fc-5675c6ebc3e1" containerName="istio-proxy" Jul 09 04:10:09.542915 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.542867 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab71f887-ef79-410a-92fc-5675c6ebc3e1" containerName="istio-proxy" Jul 09 04:10:09.542915 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.542876 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab71f887-ef79-410a-92fc-5675c6ebc3e1" containerName="istio-proxy" Jul 09 04:10:09.543075 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.542975 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab71f887-ef79-410a-92fc-5675c6ebc3e1" containerName="istio-proxy" Jul 09 04:10:09.543075 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.542991 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab71f887-ef79-410a-92fc-5675c6ebc3e1" containerName="istio-proxy" Jul 09 04:10:09.548030 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.548006 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.550878 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.550853 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a26d209c-openshift-default-dockercfg-xrp7s\"" Jul 09 04:10:09.554989 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.554965 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh"] Jul 09 04:10:09.645217 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.645183 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-podinfo\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.645217 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.645221 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-envoy\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.645489 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.645261 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-credential-socket\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.645489 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.645376 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-data\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.645603 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.645490 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b54xw\" (UniqueName: \"kubernetes.io/projected/7e2d61ab-4305-4c50-9aba-f28da3117f58-kube-api-access-b54xw\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.645603 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.645573 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-workload-certs\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.645736 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.645604 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-token\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.645736 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.645634 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-workload-socket\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.645736 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.645712 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2d61ab-4305-4c50-9aba-f28da3117f58-istiod-ca-cert\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.746739 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.746692 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b54xw\" (UniqueName: \"kubernetes.io/projected/7e2d61ab-4305-4c50-9aba-f28da3117f58-kube-api-access-b54xw\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.746958 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.746758 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-workload-certs\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.746958 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.746776 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-token\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.746958 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.746795 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-workload-socket\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.746958 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.746811 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2d61ab-4305-4c50-9aba-f28da3117f58-istiod-ca-cert\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.746958 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.746834 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-podinfo\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.746958 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.746860 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-envoy\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.746958 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.746915 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-credential-socket\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.746958 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.746955 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-data\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.747343 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.747270 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-workload-certs\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.747343 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.747306 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-workload-socket\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.747417 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.747358 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-data\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.747639 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.747618 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-credential-socket\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.747758 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.747637 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2d61ab-4305-4c50-9aba-f28da3117f58-istiod-ca-cert\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.749501 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.749482 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-envoy\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.749584 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.749563 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-podinfo\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.755147 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.755123 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-token\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.755339 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.755315 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b54xw\" (UniqueName: \"kubernetes.io/projected/7e2d61ab-4305-4c50-9aba-f28da3117f58-kube-api-access-b54xw\") pod \"e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:09.864868 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.864574 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a26d209c-openshift-default-dockercfg-xrp7s\"" Jul 09 04:10:09.872152 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:09.872125 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:10.029017 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:10.028983 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh"] Jul 09 04:10:10.032065 ip-10-0-137-230 kubenswrapper[2566]: W0709 04:10:10.032032 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e2d61ab_4305_4c50_9aba_f28da3117f58.slice/crio-e6b4d8224ef045dda27bd40266854e38b2189c76a4aad1ce684f47727985cc01 WatchSource:0}: Error finding container e6b4d8224ef045dda27bd40266854e38b2189c76a4aad1ce684f47727985cc01: Status 404 returned error can't find the container with id e6b4d8224ef045dda27bd40266854e38b2189c76a4aad1ce684f47727985cc01 Jul 09 04:10:10.034197 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:10.034164 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:10:10.034276 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:10.034236 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:10:10.034276 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:10.034262 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:10:10.520483 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:10.520441 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" event={"ID":"7e2d61ab-4305-4c50-9aba-f28da3117f58","Type":"ContainerStarted","Data":"cf2bf97934628b1f620703b83e9bd4b62b5d5ae911f21149aac0349aae8391af"} Jul 09 04:10:10.520683 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:10.520510 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" event={"ID":"7e2d61ab-4305-4c50-9aba-f28da3117f58","Type":"ContainerStarted","Data":"e6b4d8224ef045dda27bd40266854e38b2189c76a4aad1ce684f47727985cc01"} Jul 09 04:10:10.540248 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:10.540183 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" podStartSLOduration=1.540163714 podStartE2EDuration="1.540163714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:10:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:10:10.538229923 +0000 UTC m=+1681.169890010" watchObservedRunningTime="2026-07-09 04:10:10.540163714 +0000 UTC m=+1681.171823800" Jul 09 04:10:10.872818 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:10.872712 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:11.878117 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:11.878079 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:12.532286 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:12.532248 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:12.533228 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:12.533206 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:26.617794 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:26.617756 2566 generic.go:356] "Generic (PLEG): container finished" podID="7e2d61ab-4305-4c50-9aba-f28da3117f58" containerID="cf2bf97934628b1f620703b83e9bd4b62b5d5ae911f21149aac0349aae8391af" exitCode=0 Jul 09 04:10:26.618335 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:26.617799 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" event={"ID":"7e2d61ab-4305-4c50-9aba-f28da3117f58","Type":"ContainerDied","Data":"cf2bf97934628b1f620703b83e9bd4b62b5d5ae911f21149aac0349aae8391af"} Jul 09 04:10:26.618335 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:26.618281 2566 scope.go:117] "RemoveContainer" containerID="cf2bf97934628b1f620703b83e9bd4b62b5d5ae911f21149aac0349aae8391af" Jul 09 04:10:26.619336 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:26.619306 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:10:26.619428 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:26.619413 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:10:26.619490 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:26.619461 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:10:27.533012 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:27.532977 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:27.624262 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:27.624227 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" event={"ID":"7e2d61ab-4305-4c50-9aba-f28da3117f58","Type":"ContainerStarted","Data":"96f4cef9d612130d16dd5c54562e6476a9ae2410e96f6355797c2bd6b529fdc8"} Jul 09 04:10:27.874435 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:27.874336 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:27.874607 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:27.874457 2566 patch_prober.go:28] interesting pod/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 09 04:10:27.874607 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:27.874513 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" podUID="7e2d61ab-4305-4c50-9aba-f28da3117f58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 09 04:10:28.872540 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:28.872501 2566 patch_prober.go:28] interesting pod/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 09 04:10:28.873148 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:28.872567 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" podUID="7e2d61ab-4305-4c50-9aba-f28da3117f58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 09 04:10:29.877674 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:29.876760 2566 patch_prober.go:28] interesting pod/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 09 04:10:29.877674 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:29.876853 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" podUID="7e2d61ab-4305-4c50-9aba-f28da3117f58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 09 04:10:30.873545 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:30.873496 2566 patch_prober.go:28] interesting pod/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 09 04:10:30.873871 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:30.873610 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" podUID="7e2d61ab-4305-4c50-9aba-f28da3117f58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 09 04:10:31.877157 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:31.877129 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:31.877617 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:31.877395 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:10:31.878153 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:10:31.878137 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:12:08.964858 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:08.964824 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh"] Jul 09 04:12:08.965388 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:08.965041 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" podUID="7e2d61ab-4305-4c50-9aba-f28da3117f58" containerName="istio-proxy" containerID="cri-o://96f4cef9d612130d16dd5c54562e6476a9ae2410e96f6355797c2bd6b529fdc8" gracePeriod=30 Jul 09 04:12:09.961916 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:09.961884 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 04:12:09.965357 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:09.965332 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 04:12:09.966621 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:09.966602 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 04:12:09.970241 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:09.970215 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 04:12:14.184743 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.184711 2566 generic.go:356] "Generic (PLEG): container finished" podID="7e2d61ab-4305-4c50-9aba-f28da3117f58" containerID="96f4cef9d612130d16dd5c54562e6476a9ae2410e96f6355797c2bd6b529fdc8" exitCode=0 Jul 09 04:12:14.185146 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.184794 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" event={"ID":"7e2d61ab-4305-4c50-9aba-f28da3117f58","Type":"ContainerDied","Data":"96f4cef9d612130d16dd5c54562e6476a9ae2410e96f6355797c2bd6b529fdc8"} Jul 09 04:12:14.185146 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.184898 2566 scope.go:117] "RemoveContainer" containerID="cf2bf97934628b1f620703b83e9bd4b62b5d5ae911f21149aac0349aae8391af" Jul 09 04:12:14.243991 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.243928 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:12:14.324810 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.324776 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-workload-certs\") pod \"7e2d61ab-4305-4c50-9aba-f28da3117f58\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " Jul 09 04:12:14.324810 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.324811 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-credential-socket\") pod \"7e2d61ab-4305-4c50-9aba-f28da3117f58\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " Jul 09 04:12:14.325074 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.324841 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b54xw\" (UniqueName: \"kubernetes.io/projected/7e2d61ab-4305-4c50-9aba-f28da3117f58-kube-api-access-b54xw\") pod \"7e2d61ab-4305-4c50-9aba-f28da3117f58\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " Jul 09 04:12:14.325074 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.324887 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2d61ab-4305-4c50-9aba-f28da3117f58-istiod-ca-cert\") pod \"7e2d61ab-4305-4c50-9aba-f28da3117f58\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " Jul 09 04:12:14.325074 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.324930 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-workload-socket\") pod \"7e2d61ab-4305-4c50-9aba-f28da3117f58\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " Jul 09 04:12:14.325074 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.324987 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-data\") pod \"7e2d61ab-4305-4c50-9aba-f28da3117f58\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " Jul 09 04:12:14.325074 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.325021 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-podinfo\") pod \"7e2d61ab-4305-4c50-9aba-f28da3117f58\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " Jul 09 04:12:14.325074 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.325047 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-envoy\") pod \"7e2d61ab-4305-4c50-9aba-f28da3117f58\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " Jul 09 04:12:14.325074 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.325071 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-token\") pod \"7e2d61ab-4305-4c50-9aba-f28da3117f58\" (UID: \"7e2d61ab-4305-4c50-9aba-f28da3117f58\") " Jul 09 04:12:14.325438 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.325099 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7e2d61ab-4305-4c50-9aba-f28da3117f58" (UID: "7e2d61ab-4305-4c50-9aba-f28da3117f58"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:12:14.325438 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.325188 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7e2d61ab-4305-4c50-9aba-f28da3117f58" (UID: "7e2d61ab-4305-4c50-9aba-f28da3117f58"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:12:14.325438 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.325243 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7e2d61ab-4305-4c50-9aba-f28da3117f58" (UID: "7e2d61ab-4305-4c50-9aba-f28da3117f58"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:12:14.325438 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.325368 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e2d61ab-4305-4c50-9aba-f28da3117f58-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7e2d61ab-4305-4c50-9aba-f28da3117f58" (UID: "7e2d61ab-4305-4c50-9aba-f28da3117f58"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:12:14.325695 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.325458 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2d61ab-4305-4c50-9aba-f28da3117f58-istiod-ca-cert\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:12:14.325695 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.325475 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-workload-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:12:14.325695 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.325488 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-workload-certs\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:12:14.325695 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.325501 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-credential-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:12:14.325928 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.325902 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-data" (OuterVolumeSpecName: "istio-data") pod "7e2d61ab-4305-4c50-9aba-f28da3117f58" (UID: "7e2d61ab-4305-4c50-9aba-f28da3117f58"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:12:14.327183 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.327158 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e2d61ab-4305-4c50-9aba-f28da3117f58-kube-api-access-b54xw" (OuterVolumeSpecName: "kube-api-access-b54xw") pod "7e2d61ab-4305-4c50-9aba-f28da3117f58" (UID: "7e2d61ab-4305-4c50-9aba-f28da3117f58"). InnerVolumeSpecName "kube-api-access-b54xw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:12:14.327370 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.327345 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7e2d61ab-4305-4c50-9aba-f28da3117f58" (UID: "7e2d61ab-4305-4c50-9aba-f28da3117f58"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:12:14.327597 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.327569 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7e2d61ab-4305-4c50-9aba-f28da3117f58" (UID: "7e2d61ab-4305-4c50-9aba-f28da3117f58"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:12:14.327744 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.327666 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e2d61ab-4305-4c50-9aba-f28da3117f58" (UID: "7e2d61ab-4305-4c50-9aba-f28da3117f58"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:12:14.426060 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.426005 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-data\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:12:14.426060 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.426055 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-podinfo\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:12:14.426060 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.426065 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-envoy\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:12:14.426060 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.426073 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2d61ab-4305-4c50-9aba-f28da3117f58-istio-token\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:12:14.426060 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:14.426082 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b54xw\" (UniqueName: \"kubernetes.io/projected/7e2d61ab-4305-4c50-9aba-f28da3117f58-kube-api-access-b54xw\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:12:15.191048 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:15.191018 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" Jul 09 04:12:15.191499 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:15.191052 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh" event={"ID":"7e2d61ab-4305-4c50-9aba-f28da3117f58","Type":"ContainerDied","Data":"e6b4d8224ef045dda27bd40266854e38b2189c76a4aad1ce684f47727985cc01"} Jul 09 04:12:15.191499 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:15.191109 2566 scope.go:117] "RemoveContainer" containerID="96f4cef9d612130d16dd5c54562e6476a9ae2410e96f6355797c2bd6b529fdc8" Jul 09 04:12:15.215032 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:15.215010 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh"] Jul 09 04:12:15.218332 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:15.218311 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a26d209c-openshift-default-6967fdf79-7vwvh"] Jul 09 04:12:15.865464 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:12:15.865430 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e2d61ab-4305-4c50-9aba-f28da3117f58" path="/var/lib/kubelet/pods/7e2d61ab-4305-4c50-9aba-f28da3117f58/volumes" Jul 09 04:13:03.182321 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.182286 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj"] Jul 09 04:13:03.182984 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.182963 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e2d61ab-4305-4c50-9aba-f28da3117f58" containerName="istio-proxy" Jul 09 04:13:03.183039 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.182987 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e2d61ab-4305-4c50-9aba-f28da3117f58" containerName="istio-proxy" Jul 09 04:13:03.183039 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.183026 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e2d61ab-4305-4c50-9aba-f28da3117f58" containerName="istio-proxy" Jul 09 04:13:03.183039 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.183034 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e2d61ab-4305-4c50-9aba-f28da3117f58" containerName="istio-proxy" Jul 09 04:13:03.183162 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.183149 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e2d61ab-4305-4c50-9aba-f28da3117f58" containerName="istio-proxy" Jul 09 04:13:03.183200 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.183168 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e2d61ab-4305-4c50-9aba-f28da3117f58" containerName="istio-proxy" Jul 09 04:13:03.186810 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.186791 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.189371 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.189351 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-2eb6ca-openshift-default-dockercfg-gm6cx\"" Jul 09 04:13:03.197708 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.197682 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj"] Jul 09 04:13:03.269811 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.269772 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-workload-certs\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.269982 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.269836 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-istio-data\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.269982 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.269879 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-istio-envoy\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.269982 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.269951 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43207361-15b0-4055-98ef-4ba22567837e-istio-token\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.270103 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.269982 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43207361-15b0-4055-98ef-4ba22567837e-istio-podinfo\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.270103 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.270006 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-workload-socket\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.270103 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.270020 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-credential-socket\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.270103 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.270039 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43207361-15b0-4055-98ef-4ba22567837e-istiod-ca-cert\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.270103 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.270056 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cvbn\" (UniqueName: \"kubernetes.io/projected/43207361-15b0-4055-98ef-4ba22567837e-kube-api-access-8cvbn\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.370521 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.370486 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43207361-15b0-4055-98ef-4ba22567837e-istio-token\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.370736 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.370530 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43207361-15b0-4055-98ef-4ba22567837e-istio-podinfo\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.370736 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.370661 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-workload-socket\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.370736 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.370708 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-credential-socket\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.370736 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.370733 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43207361-15b0-4055-98ef-4ba22567837e-istiod-ca-cert\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.370967 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.370751 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cvbn\" (UniqueName: \"kubernetes.io/projected/43207361-15b0-4055-98ef-4ba22567837e-kube-api-access-8cvbn\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.370967 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.370837 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-workload-certs\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.370967 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.370903 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-istio-data\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.371136 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.370972 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-istio-envoy\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.371136 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.371069 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-workload-socket\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.371136 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.371106 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-credential-socket\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.371477 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.371452 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-istio-data\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.371617 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.371471 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43207361-15b0-4055-98ef-4ba22567837e-istiod-ca-cert\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.371617 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.371593 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-workload-certs\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.373228 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.373206 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43207361-15b0-4055-98ef-4ba22567837e-istio-podinfo\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.373322 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.373248 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-istio-envoy\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.379227 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.379192 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43207361-15b0-4055-98ef-4ba22567837e-istio-token\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.379384 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.379365 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cvbn\" (UniqueName: \"kubernetes.io/projected/43207361-15b0-4055-98ef-4ba22567837e-kube-api-access-8cvbn\") pod \"e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.500922 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.500828 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:03.660800 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.660777 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj"] Jul 09 04:13:03.663041 ip-10-0-137-230 kubenswrapper[2566]: W0709 04:13:03.663015 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43207361_15b0_4055_98ef_4ba22567837e.slice/crio-b3d039d8dfd98b65820f4151c74b74840226f6a8195489d4eac1a972033fb1c5 WatchSource:0}: Error finding container b3d039d8dfd98b65820f4151c74b74840226f6a8195489d4eac1a972033fb1c5: Status 404 returned error can't find the container with id b3d039d8dfd98b65820f4151c74b74840226f6a8195489d4eac1a972033fb1c5 Jul 09 04:13:03.665046 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.665013 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:13:03.665134 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.665080 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:13:03.665134 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:03.665113 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:13:04.438266 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:04.438226 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" event={"ID":"43207361-15b0-4055-98ef-4ba22567837e","Type":"ContainerStarted","Data":"510acaa73ef96f1a25ee44040d7ec7c47741dc6fb636ed2fc29a965bc274b2a0"} Jul 09 04:13:04.438801 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:04.438287 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" event={"ID":"43207361-15b0-4055-98ef-4ba22567837e","Type":"ContainerStarted","Data":"b3d039d8dfd98b65820f4151c74b74840226f6a8195489d4eac1a972033fb1c5"} Jul 09 04:13:04.459333 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:04.459282 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" podStartSLOduration=1.459267429 podStartE2EDuration="1.459267429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:13:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:13:04.457531445 +0000 UTC m=+1855.089191533" watchObservedRunningTime="2026-07-09 04:13:04.459267429 +0000 UTC m=+1855.090927568" Jul 09 04:13:04.501878 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:04.501840 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:04.506904 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:04.506881 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:05.443667 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:05.443603 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:05.444773 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:05.444751 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:11.411758 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.411718 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42"] Jul 09 04:13:11.417719 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.417684 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.420327 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.420304 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-2dbf5f-openshift-default-dockercfg-sqpdx\"" Jul 09 04:13:11.427800 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.427768 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42"] Jul 09 04:13:11.544950 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.544908 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-credential-socket\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.545159 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.544967 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-data\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.545159 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.544990 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvvxf\" (UniqueName: \"kubernetes.io/projected/c2a61d84-b931-45b8-9f97-3912efb6ce2a-kube-api-access-bvvxf\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.545159 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.545133 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istiod-ca-cert\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.545339 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.545176 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-podinfo\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.545339 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.545215 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-workload-certs\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.545339 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.545253 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-envoy\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.545339 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.545286 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-workload-socket\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.545339 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.545311 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-token\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.646899 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.646857 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-envoy\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.646899 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.646903 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-workload-socket\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.647154 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.646923 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-token\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.647154 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.646959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-credential-socket\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.647154 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.646998 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-data\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.647154 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.647013 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvvxf\" (UniqueName: \"kubernetes.io/projected/c2a61d84-b931-45b8-9f97-3912efb6ce2a-kube-api-access-bvvxf\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.647154 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.647090 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istiod-ca-cert\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.647154 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.647114 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-podinfo\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.647557 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.647390 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-workload-socket\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.647557 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.647416 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-workload-certs\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.647557 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.647433 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-credential-socket\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.647557 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.647491 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-data\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.647806 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.647783 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-workload-certs\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.647925 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.647901 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istiod-ca-cert\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.649343 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.649317 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-envoy\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.649727 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.649711 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-podinfo\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.655963 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.655939 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-token\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.656062 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.655994 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvvxf\" (UniqueName: \"kubernetes.io/projected/c2a61d84-b931-45b8-9f97-3912efb6ce2a-kube-api-access-bvvxf\") pod \"e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.729281 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.729164 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:11.878889 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.878768 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42"] Jul 09 04:13:11.881328 ip-10-0-137-230 kubenswrapper[2566]: W0709 04:13:11.881292 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2a61d84_b931_45b8_9f97_3912efb6ce2a.slice/crio-5ece4b878ce75893aa216456fb976021283d774c21464fcdaee0ce0cbd0d67d6 WatchSource:0}: Error finding container 5ece4b878ce75893aa216456fb976021283d774c21464fcdaee0ce0cbd0d67d6: Status 404 returned error can't find the container with id 5ece4b878ce75893aa216456fb976021283d774c21464fcdaee0ce0cbd0d67d6 Jul 09 04:13:11.884021 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.883985 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:13:11.884135 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.884056 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:13:11.884135 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:11.884084 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:13:12.481803 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:12.481758 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" event={"ID":"c2a61d84-b931-45b8-9f97-3912efb6ce2a","Type":"ContainerStarted","Data":"8a03435e8d446b227cf3c519ffc9943b906a81b85a8ea0551e041b5f3428e208"} Jul 09 04:13:12.482241 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:12.481849 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" event={"ID":"c2a61d84-b931-45b8-9f97-3912efb6ce2a","Type":"ContainerStarted","Data":"5ece4b878ce75893aa216456fb976021283d774c21464fcdaee0ce0cbd0d67d6"} Jul 09 04:13:12.502450 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:12.502398 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" podStartSLOduration=1.50238074 podStartE2EDuration="1.50238074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:13:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:13:12.501024624 +0000 UTC m=+1863.132684726" watchObservedRunningTime="2026-07-09 04:13:12.50238074 +0000 UTC m=+1863.134040826" Jul 09 04:13:12.729839 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:12.729795 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:12.734879 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:12.734801 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:13.487154 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:13.487124 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:13.488285 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:13.488269 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:21.486875 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:21.486829 2566 patch_prober.go:28] interesting pod/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 04:13:21.487304 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:21.486909 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" podUID="43207361-15b0-4055-98ef-4ba22567837e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 04:13:22.540925 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:22.540851 2566 generic.go:356] "Generic (PLEG): container finished" podID="43207361-15b0-4055-98ef-4ba22567837e" containerID="510acaa73ef96f1a25ee44040d7ec7c47741dc6fb636ed2fc29a965bc274b2a0" exitCode=0 Jul 09 04:13:22.541328 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:22.540923 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" event={"ID":"43207361-15b0-4055-98ef-4ba22567837e","Type":"ContainerDied","Data":"510acaa73ef96f1a25ee44040d7ec7c47741dc6fb636ed2fc29a965bc274b2a0"} Jul 09 04:13:22.541373 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:22.541334 2566 scope.go:117] "RemoveContainer" containerID="510acaa73ef96f1a25ee44040d7ec7c47741dc6fb636ed2fc29a965bc274b2a0" Jul 09 04:13:22.542369 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:22.542338 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:13:22.542474 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:22.542421 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:13:22.542538 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:22.542473 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:13:23.548629 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:23.548596 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" event={"ID":"43207361-15b0-4055-98ef-4ba22567837e","Type":"ContainerStarted","Data":"a494733e034f3e5f65977c11f250f61092bf517caa6246dc46424ebee89d8803"} Jul 09 04:13:24.502585 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:24.502098 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:24.502585 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:24.502239 2566 patch_prober.go:28] interesting pod/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 09 04:13:24.502585 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:24.502285 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" podUID="43207361-15b0-4055-98ef-4ba22567837e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 09 04:13:25.502068 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:25.502026 2566 patch_prober.go:28] interesting pod/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 09 04:13:25.502738 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:25.502694 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" podUID="43207361-15b0-4055-98ef-4ba22567837e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 09 04:13:26.502090 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:26.501605 2566 patch_prober.go:28] interesting pod/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 09 04:13:26.502090 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:26.501692 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" podUID="43207361-15b0-4055-98ef-4ba22567837e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 09 04:13:27.501899 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:27.501855 2566 patch_prober.go:28] interesting pod/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 09 04:13:27.502151 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:27.501915 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" podUID="43207361-15b0-4055-98ef-4ba22567837e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 09 04:13:28.505190 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:28.505163 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:28.505590 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:28.505424 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:28.506179 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:28.506160 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:13:29.170119 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:29.170081 2566 patch_prober.go:28] interesting pod/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": read tcp 10.132.0.2:41072->10.132.0.57:15021: read: connection reset by peer" start-of-body= Jul 09 04:13:29.170299 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:29.170160 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" podUID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": read tcp 10.132.0.2:41072->10.132.0.57:15021: read: connection reset by peer" Jul 09 04:13:29.584126 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:29.584032 2566 generic.go:356] "Generic (PLEG): container finished" podID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerID="8a03435e8d446b227cf3c519ffc9943b906a81b85a8ea0551e041b5f3428e208" exitCode=0 Jul 09 04:13:29.584126 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:29.584106 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" event={"ID":"c2a61d84-b931-45b8-9f97-3912efb6ce2a","Type":"ContainerDied","Data":"8a03435e8d446b227cf3c519ffc9943b906a81b85a8ea0551e041b5f3428e208"} Jul 09 04:13:29.584704 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:29.584686 2566 scope.go:117] "RemoveContainer" containerID="8a03435e8d446b227cf3c519ffc9943b906a81b85a8ea0551e041b5f3428e208" Jul 09 04:13:29.585671 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:29.585628 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:13:29.585787 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:29.585706 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:13:29.585787 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:29.585742 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:13:30.597245 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:30.596169 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" event={"ID":"c2a61d84-b931-45b8-9f97-3912efb6ce2a","Type":"ContainerStarted","Data":"36a70da1434982ac09cb1f2ef7f03406131e69fc2d0d0ced4432f25f42315d1b"} Jul 09 04:13:30.730283 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:30.730240 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:30.730542 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:30.730521 2566 patch_prober.go:28] interesting pod/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 04:13:30.730600 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:30.730582 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" podUID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 04:13:31.729964 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:31.729930 2566 patch_prober.go:28] interesting pod/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 04:13:31.730352 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:31.729985 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" podUID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 04:13:32.730462 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:32.730369 2566 patch_prober.go:28] interesting pod/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 04:13:32.730462 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:32.730430 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" podUID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 04:13:33.730288 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:33.730258 2566 patch_prober.go:28] interesting pod/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 04:13:33.730466 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:33.730311 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" podUID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 04:13:34.733881 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:34.733849 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:34.734280 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:34.734189 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:13:34.740688 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:13:34.735295 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:14:55.570373 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:14:55.570333 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj"] Jul 09 04:14:55.570918 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:14:55.570608 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" podUID="43207361-15b0-4055-98ef-4ba22567837e" containerName="istio-proxy" containerID="cri-o://a494733e034f3e5f65977c11f250f61092bf517caa6246dc46424ebee89d8803" gracePeriod=30 Jul 09 04:14:58.508319 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:14:58.508280 2566 patch_prober.go:28] interesting pod/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 04:14:58.508750 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:14:58.508352 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" podUID="43207361-15b0-4055-98ef-4ba22567837e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 04:15:00.837903 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.837833 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:15:00.920908 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.920873 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-credential-socket\") pod \"43207361-15b0-4055-98ef-4ba22567837e\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " Jul 09 04:15:00.920908 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.920920 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43207361-15b0-4055-98ef-4ba22567837e-istio-token\") pod \"43207361-15b0-4055-98ef-4ba22567837e\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " Jul 09 04:15:00.921185 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.920956 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-istio-envoy\") pod \"43207361-15b0-4055-98ef-4ba22567837e\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " Jul 09 04:15:00.921185 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.920989 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-workload-certs\") pod \"43207361-15b0-4055-98ef-4ba22567837e\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " Jul 09 04:15:00.921185 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.921015 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-istio-data\") pod \"43207361-15b0-4055-98ef-4ba22567837e\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " Jul 09 04:15:00.921185 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.921124 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-workload-socket\") pod \"43207361-15b0-4055-98ef-4ba22567837e\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " Jul 09 04:15:00.921410 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.921191 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43207361-15b0-4055-98ef-4ba22567837e-istio-podinfo\") pod \"43207361-15b0-4055-98ef-4ba22567837e\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " Jul 09 04:15:00.921410 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.921220 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cvbn\" (UniqueName: \"kubernetes.io/projected/43207361-15b0-4055-98ef-4ba22567837e-kube-api-access-8cvbn\") pod \"43207361-15b0-4055-98ef-4ba22567837e\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " Jul 09 04:15:00.921410 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.921240 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "43207361-15b0-4055-98ef-4ba22567837e" (UID: "43207361-15b0-4055-98ef-4ba22567837e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:00.921410 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.921283 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43207361-15b0-4055-98ef-4ba22567837e-istiod-ca-cert\") pod \"43207361-15b0-4055-98ef-4ba22567837e\" (UID: \"43207361-15b0-4055-98ef-4ba22567837e\") " Jul 09 04:15:00.921610 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.921407 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "43207361-15b0-4055-98ef-4ba22567837e" (UID: "43207361-15b0-4055-98ef-4ba22567837e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:00.921703 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.921595 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "43207361-15b0-4055-98ef-4ba22567837e" (UID: "43207361-15b0-4055-98ef-4ba22567837e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:00.921773 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.921750 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-workload-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:00.921831 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.921763 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43207361-15b0-4055-98ef-4ba22567837e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "43207361-15b0-4055-98ef-4ba22567837e" (UID: "43207361-15b0-4055-98ef-4ba22567837e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:15:00.921831 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.921773 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-credential-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:00.921831 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.921807 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-workload-certs\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:00.922041 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.922020 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-istio-data" (OuterVolumeSpecName: "istio-data") pod "43207361-15b0-4055-98ef-4ba22567837e" (UID: "43207361-15b0-4055-98ef-4ba22567837e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:00.923544 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.923514 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/43207361-15b0-4055-98ef-4ba22567837e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "43207361-15b0-4055-98ef-4ba22567837e" (UID: "43207361-15b0-4055-98ef-4ba22567837e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:15:00.923544 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.923539 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "43207361-15b0-4055-98ef-4ba22567837e" (UID: "43207361-15b0-4055-98ef-4ba22567837e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:00.923836 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.923815 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43207361-15b0-4055-98ef-4ba22567837e-istio-token" (OuterVolumeSpecName: "istio-token") pod "43207361-15b0-4055-98ef-4ba22567837e" (UID: "43207361-15b0-4055-98ef-4ba22567837e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:15:00.923897 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:00.923832 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43207361-15b0-4055-98ef-4ba22567837e-kube-api-access-8cvbn" (OuterVolumeSpecName: "kube-api-access-8cvbn") pod "43207361-15b0-4055-98ef-4ba22567837e" (UID: "43207361-15b0-4055-98ef-4ba22567837e"). InnerVolumeSpecName "kube-api-access-8cvbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:15:01.022256 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.022212 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43207361-15b0-4055-98ef-4ba22567837e-istio-podinfo\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:01.022256 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.022249 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cvbn\" (UniqueName: \"kubernetes.io/projected/43207361-15b0-4055-98ef-4ba22567837e-kube-api-access-8cvbn\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:01.022256 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.022262 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43207361-15b0-4055-98ef-4ba22567837e-istiod-ca-cert\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:01.022509 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.022273 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43207361-15b0-4055-98ef-4ba22567837e-istio-token\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:01.022509 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.022282 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-istio-envoy\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:01.022509 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.022289 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43207361-15b0-4055-98ef-4ba22567837e-istio-data\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:01.071416 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.071377 2566 generic.go:356] "Generic (PLEG): container finished" podID="43207361-15b0-4055-98ef-4ba22567837e" containerID="a494733e034f3e5f65977c11f250f61092bf517caa6246dc46424ebee89d8803" exitCode=0 Jul 09 04:15:01.071594 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.071436 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" Jul 09 04:15:01.071594 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.071465 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" event={"ID":"43207361-15b0-4055-98ef-4ba22567837e","Type":"ContainerDied","Data":"a494733e034f3e5f65977c11f250f61092bf517caa6246dc46424ebee89d8803"} Jul 09 04:15:01.071594 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.071529 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj" event={"ID":"43207361-15b0-4055-98ef-4ba22567837e","Type":"ContainerDied","Data":"b3d039d8dfd98b65820f4151c74b74840226f6a8195489d4eac1a972033fb1c5"} Jul 09 04:15:01.071594 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.071558 2566 scope.go:117] "RemoveContainer" containerID="a494733e034f3e5f65977c11f250f61092bf517caa6246dc46424ebee89d8803" Jul 09 04:15:01.081753 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.081731 2566 scope.go:117] "RemoveContainer" containerID="510acaa73ef96f1a25ee44040d7ec7c47741dc6fb636ed2fc29a965bc274b2a0" Jul 09 04:15:01.091046 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.091025 2566 scope.go:117] "RemoveContainer" containerID="a494733e034f3e5f65977c11f250f61092bf517caa6246dc46424ebee89d8803" Jul 09 04:15:01.091361 ip-10-0-137-230 kubenswrapper[2566]: E0709 04:15:01.091343 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a494733e034f3e5f65977c11f250f61092bf517caa6246dc46424ebee89d8803\": container with ID starting with a494733e034f3e5f65977c11f250f61092bf517caa6246dc46424ebee89d8803 not found: ID does not exist" containerID="a494733e034f3e5f65977c11f250f61092bf517caa6246dc46424ebee89d8803" Jul 09 04:15:01.091416 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.091370 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a494733e034f3e5f65977c11f250f61092bf517caa6246dc46424ebee89d8803"} err="failed to get container status \"a494733e034f3e5f65977c11f250f61092bf517caa6246dc46424ebee89d8803\": rpc error: code = NotFound desc = could not find container \"a494733e034f3e5f65977c11f250f61092bf517caa6246dc46424ebee89d8803\": container with ID starting with a494733e034f3e5f65977c11f250f61092bf517caa6246dc46424ebee89d8803 not found: ID does not exist" Jul 09 04:15:01.091416 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.091392 2566 scope.go:117] "RemoveContainer" containerID="510acaa73ef96f1a25ee44040d7ec7c47741dc6fb636ed2fc29a965bc274b2a0" Jul 09 04:15:01.091635 ip-10-0-137-230 kubenswrapper[2566]: E0709 04:15:01.091609 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"510acaa73ef96f1a25ee44040d7ec7c47741dc6fb636ed2fc29a965bc274b2a0\": container with ID starting with 510acaa73ef96f1a25ee44040d7ec7c47741dc6fb636ed2fc29a965bc274b2a0 not found: ID does not exist" containerID="510acaa73ef96f1a25ee44040d7ec7c47741dc6fb636ed2fc29a965bc274b2a0" Jul 09 04:15:01.091722 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.091664 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"510acaa73ef96f1a25ee44040d7ec7c47741dc6fb636ed2fc29a965bc274b2a0"} err="failed to get container status \"510acaa73ef96f1a25ee44040d7ec7c47741dc6fb636ed2fc29a965bc274b2a0\": rpc error: code = NotFound desc = could not find container \"510acaa73ef96f1a25ee44040d7ec7c47741dc6fb636ed2fc29a965bc274b2a0\": container with ID starting with 510acaa73ef96f1a25ee44040d7ec7c47741dc6fb636ed2fc29a965bc274b2a0 not found: ID does not exist" Jul 09 04:15:01.094933 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.094906 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj"] Jul 09 04:15:01.098962 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.098931 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-2eb6ca-openshift-default-698b545b59-zk9hj"] Jul 09 04:15:01.865472 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:01.865437 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43207361-15b0-4055-98ef-4ba22567837e" path="/var/lib/kubelet/pods/43207361-15b0-4055-98ef-4ba22567837e/volumes" Jul 09 04:15:02.716346 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:02.716244 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42"] Jul 09 04:15:02.716544 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:02.716498 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" podUID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerName="istio-proxy" containerID="cri-o://36a70da1434982ac09cb1f2ef7f03406131e69fc2d0d0ced4432f25f42315d1b" gracePeriod=30 Jul 09 04:15:04.737329 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:04.737293 2566 patch_prober.go:28] interesting pod/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 04:15:04.737747 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:04.737351 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" podUID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 04:15:07.979491 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:07.979429 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:15:08.084033 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.083989 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-podinfo\") pod \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " Jul 09 04:15:08.084033 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084037 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-workload-certs\") pod \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " Jul 09 04:15:08.084310 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084074 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvvxf\" (UniqueName: \"kubernetes.io/projected/c2a61d84-b931-45b8-9f97-3912efb6ce2a-kube-api-access-bvvxf\") pod \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " Jul 09 04:15:08.084310 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084114 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-credential-socket\") pod \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " Jul 09 04:15:08.084310 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084196 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istiod-ca-cert\") pod \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " Jul 09 04:15:08.084310 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084261 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-token\") pod \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " Jul 09 04:15:08.084310 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084298 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-workload-socket\") pod \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " Jul 09 04:15:08.084564 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084330 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-data\") pod \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " Jul 09 04:15:08.084564 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084363 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-envoy\") pod \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\" (UID: \"c2a61d84-b931-45b8-9f97-3912efb6ce2a\") " Jul 09 04:15:08.084564 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084426 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c2a61d84-b931-45b8-9f97-3912efb6ce2a" (UID: "c2a61d84-b931-45b8-9f97-3912efb6ce2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:08.084564 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084437 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c2a61d84-b931-45b8-9f97-3912efb6ce2a" (UID: "c2a61d84-b931-45b8-9f97-3912efb6ce2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:08.084793 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084696 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c2a61d84-b931-45b8-9f97-3912efb6ce2a" (UID: "c2a61d84-b931-45b8-9f97-3912efb6ce2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:08.084793 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084730 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c2a61d84-b931-45b8-9f97-3912efb6ce2a" (UID: "c2a61d84-b931-45b8-9f97-3912efb6ce2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:15:08.084913 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084802 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-workload-certs\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:08.084913 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084821 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-credential-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:08.084913 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.084837 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-workload-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:08.085449 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.085424 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-data" (OuterVolumeSpecName: "istio-data") pod "c2a61d84-b931-45b8-9f97-3912efb6ce2a" (UID: "c2a61d84-b931-45b8-9f97-3912efb6ce2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:08.086504 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.086469 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2a61d84-b931-45b8-9f97-3912efb6ce2a-kube-api-access-bvvxf" (OuterVolumeSpecName: "kube-api-access-bvvxf") pod "c2a61d84-b931-45b8-9f97-3912efb6ce2a" (UID: "c2a61d84-b931-45b8-9f97-3912efb6ce2a"). InnerVolumeSpecName "kube-api-access-bvvxf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:15:08.086603 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.086521 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-token" (OuterVolumeSpecName: "istio-token") pod "c2a61d84-b931-45b8-9f97-3912efb6ce2a" (UID: "c2a61d84-b931-45b8-9f97-3912efb6ce2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:15:08.086726 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.086706 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c2a61d84-b931-45b8-9f97-3912efb6ce2a" (UID: "c2a61d84-b931-45b8-9f97-3912efb6ce2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:15:08.086826 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.086807 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c2a61d84-b931-45b8-9f97-3912efb6ce2a" (UID: "c2a61d84-b931-45b8-9f97-3912efb6ce2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:08.111457 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.111428 2566 generic.go:356] "Generic (PLEG): container finished" podID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerID="36a70da1434982ac09cb1f2ef7f03406131e69fc2d0d0ced4432f25f42315d1b" exitCode=0 Jul 09 04:15:08.111593 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.111478 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" event={"ID":"c2a61d84-b931-45b8-9f97-3912efb6ce2a","Type":"ContainerDied","Data":"36a70da1434982ac09cb1f2ef7f03406131e69fc2d0d0ced4432f25f42315d1b"} Jul 09 04:15:08.111593 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.111517 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" event={"ID":"c2a61d84-b931-45b8-9f97-3912efb6ce2a","Type":"ContainerDied","Data":"5ece4b878ce75893aa216456fb976021283d774c21464fcdaee0ce0cbd0d67d6"} Jul 09 04:15:08.111593 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.111538 2566 scope.go:117] "RemoveContainer" containerID="36a70da1434982ac09cb1f2ef7f03406131e69fc2d0d0ced4432f25f42315d1b" Jul 09 04:15:08.111593 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.111480 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42" Jul 09 04:15:08.125804 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.125786 2566 scope.go:117] "RemoveContainer" containerID="8a03435e8d446b227cf3c519ffc9943b906a81b85a8ea0551e041b5f3428e208" Jul 09 04:15:08.133825 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.133796 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42"] Jul 09 04:15:08.135125 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.135095 2566 scope.go:117] "RemoveContainer" containerID="36a70da1434982ac09cb1f2ef7f03406131e69fc2d0d0ced4432f25f42315d1b" Jul 09 04:15:08.135623 ip-10-0-137-230 kubenswrapper[2566]: E0709 04:15:08.135582 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36a70da1434982ac09cb1f2ef7f03406131e69fc2d0d0ced4432f25f42315d1b\": container with ID starting with 36a70da1434982ac09cb1f2ef7f03406131e69fc2d0d0ced4432f25f42315d1b not found: ID does not exist" containerID="36a70da1434982ac09cb1f2ef7f03406131e69fc2d0d0ced4432f25f42315d1b" Jul 09 04:15:08.135755 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.135634 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36a70da1434982ac09cb1f2ef7f03406131e69fc2d0d0ced4432f25f42315d1b"} err="failed to get container status \"36a70da1434982ac09cb1f2ef7f03406131e69fc2d0d0ced4432f25f42315d1b\": rpc error: code = NotFound desc = could not find container \"36a70da1434982ac09cb1f2ef7f03406131e69fc2d0d0ced4432f25f42315d1b\": container with ID starting with 36a70da1434982ac09cb1f2ef7f03406131e69fc2d0d0ced4432f25f42315d1b not found: ID does not exist" Jul 09 04:15:08.135755 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.135681 2566 scope.go:117] "RemoveContainer" containerID="8a03435e8d446b227cf3c519ffc9943b906a81b85a8ea0551e041b5f3428e208" Jul 09 04:15:08.136039 ip-10-0-137-230 kubenswrapper[2566]: E0709 04:15:08.136015 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a03435e8d446b227cf3c519ffc9943b906a81b85a8ea0551e041b5f3428e208\": container with ID starting with 8a03435e8d446b227cf3c519ffc9943b906a81b85a8ea0551e041b5f3428e208 not found: ID does not exist" containerID="8a03435e8d446b227cf3c519ffc9943b906a81b85a8ea0551e041b5f3428e208" Jul 09 04:15:08.136137 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.136050 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a03435e8d446b227cf3c519ffc9943b906a81b85a8ea0551e041b5f3428e208"} err="failed to get container status \"8a03435e8d446b227cf3c519ffc9943b906a81b85a8ea0551e041b5f3428e208\": rpc error: code = NotFound desc = could not find container \"8a03435e8d446b227cf3c519ffc9943b906a81b85a8ea0551e041b5f3428e208\": container with ID starting with 8a03435e8d446b227cf3c519ffc9943b906a81b85a8ea0551e041b5f3428e208 not found: ID does not exist" Jul 09 04:15:08.137566 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.137546 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-2dbf5f-openshift-default-684d8f-jbz42"] Jul 09 04:15:08.186122 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.186087 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-data\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:08.186122 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.186114 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-envoy\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:08.186122 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.186123 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-podinfo\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:08.186335 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.186132 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvvxf\" (UniqueName: \"kubernetes.io/projected/c2a61d84-b931-45b8-9f97-3912efb6ce2a-kube-api-access-bvvxf\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:08.186335 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.186145 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istiod-ca-cert\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:08.186335 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:08.186155 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2a61d84-b931-45b8-9f97-3912efb6ce2a-istio-token\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:09.867720 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:09.867687 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" path="/var/lib/kubelet/pods/c2a61d84-b931-45b8-9f97-3912efb6ce2a/volumes" Jul 09 04:15:16.079250 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.079210 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m"] Jul 09 04:15:16.079838 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.079822 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43207361-15b0-4055-98ef-4ba22567837e" containerName="istio-proxy" Jul 09 04:15:16.079899 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.079841 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="43207361-15b0-4055-98ef-4ba22567837e" containerName="istio-proxy" Jul 09 04:15:16.079899 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.079858 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerName="istio-proxy" Jul 09 04:15:16.079899 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.079867 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerName="istio-proxy" Jul 09 04:15:16.079988 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.079979 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerName="istio-proxy" Jul 09 04:15:16.080026 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.079991 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="43207361-15b0-4055-98ef-4ba22567837e" containerName="istio-proxy" Jul 09 04:15:16.080026 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.080000 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerName="istio-proxy" Jul 09 04:15:16.080026 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.080013 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="43207361-15b0-4055-98ef-4ba22567837e" containerName="istio-proxy" Jul 09 04:15:16.082565 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.082543 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.085496 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.085469 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f007b3-openshift-default-dockercfg-rmkcr\"" Jul 09 04:15:16.093521 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.093490 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m"] Jul 09 04:15:16.165718 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.165639 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-envoy\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.165718 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.165724 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-token\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.165999 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.165786 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-data\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.165999 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.165833 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w74lt\" (UniqueName: \"kubernetes.io/projected/1e6c6cc6-c6c0-4550-b972-85ac09af5082-kube-api-access-w74lt\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.165999 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.165857 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-podinfo\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.165999 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.165928 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-workload-certs\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.165999 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.165962 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-workload-socket\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.165999 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.165988 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istiod-ca-cert\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.166243 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.166022 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-credential-socket\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.267663 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.267593 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-envoy\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.267877 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.267759 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-token\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.267938 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.267881 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-data\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.267997 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.267935 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w74lt\" (UniqueName: \"kubernetes.io/projected/1e6c6cc6-c6c0-4550-b972-85ac09af5082-kube-api-access-w74lt\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.267997 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.267962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-podinfo\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.268111 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.268015 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-workload-certs\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.268111 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.268050 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-workload-socket\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.268111 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.268074 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istiod-ca-cert\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.268111 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.268110 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-credential-socket\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.268309 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.268274 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-data\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.268416 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.268377 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-workload-certs\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.268499 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.268448 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-credential-socket\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.268553 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.268527 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-workload-socket\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.268929 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.268906 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istiod-ca-cert\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.270230 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.270200 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-envoy\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.270460 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.270440 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-podinfo\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.279965 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.279925 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-token\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.280132 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.280037 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w74lt\" (UniqueName: \"kubernetes.io/projected/1e6c6cc6-c6c0-4550-b972-85ac09af5082-kube-api-access-w74lt\") pod \"e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.396119 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.396081 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:16.543659 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.543615 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m"] Jul 09 04:15:16.545583 ip-10-0-137-230 kubenswrapper[2566]: W0709 04:15:16.545553 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e6c6cc6_c6c0_4550_b972_85ac09af5082.slice/crio-da7a90e6aa4c5db5483a61b7b0995065b7b81a5cd98fdd405900745a53b01d74 WatchSource:0}: Error finding container da7a90e6aa4c5db5483a61b7b0995065b7b81a5cd98fdd405900745a53b01d74: Status 404 returned error can't find the container with id da7a90e6aa4c5db5483a61b7b0995065b7b81a5cd98fdd405900745a53b01d74 Jul 09 04:15:16.548084 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.548050 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:15:16.548169 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.548119 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:15:16.548169 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:16.548149 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:15:17.163122 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:17.163071 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" event={"ID":"1e6c6cc6-c6c0-4550-b972-85ac09af5082","Type":"ContainerStarted","Data":"e9fe406cc275f1fd8638252fe55a4a5c32573f0cfd510aed7346073c286ad914"} Jul 09 04:15:17.163534 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:17.163153 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" event={"ID":"1e6c6cc6-c6c0-4550-b972-85ac09af5082","Type":"ContainerStarted","Data":"da7a90e6aa4c5db5483a61b7b0995065b7b81a5cd98fdd405900745a53b01d74"} Jul 09 04:15:17.189663 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:17.189593 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" podStartSLOduration=1.189574718 podStartE2EDuration="1.189574718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:15:17.187524239 +0000 UTC m=+1987.819184326" watchObservedRunningTime="2026-07-09 04:15:17.189574718 +0000 UTC m=+1987.821234803" Jul 09 04:15:17.397157 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:17.397114 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:17.402297 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:17.402271 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:18.167992 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:18.167960 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:18.168935 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:18.168916 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:32.242092 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:32.242046 2566 generic.go:356] "Generic (PLEG): container finished" podID="1e6c6cc6-c6c0-4550-b972-85ac09af5082" containerID="e9fe406cc275f1fd8638252fe55a4a5c32573f0cfd510aed7346073c286ad914" exitCode=0 Jul 09 04:15:32.242522 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:32.242119 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" event={"ID":"1e6c6cc6-c6c0-4550-b972-85ac09af5082","Type":"ContainerDied","Data":"e9fe406cc275f1fd8638252fe55a4a5c32573f0cfd510aed7346073c286ad914"} Jul 09 04:15:32.242565 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:32.242538 2566 scope.go:117] "RemoveContainer" containerID="e9fe406cc275f1fd8638252fe55a4a5c32573f0cfd510aed7346073c286ad914" Jul 09 04:15:32.243598 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:32.243572 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:15:32.243689 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:32.243674 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:15:32.243735 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:32.243722 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:15:33.168298 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:33.168257 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:33.248639 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:33.248594 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" event={"ID":"1e6c6cc6-c6c0-4550-b972-85ac09af5082","Type":"ContainerStarted","Data":"0f12fc73050e1030daca6db43fe8365cc0c709ea8dbfcf0696703efc672f86c0"} Jul 09 04:15:33.396720 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:33.396678 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:33.398415 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:33.398380 2566 patch_prober.go:28] interesting pod/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 09 04:15:33.398590 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:33.398444 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" podUID="1e6c6cc6-c6c0-4550-b972-85ac09af5082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 09 04:15:34.396880 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:34.396834 2566 patch_prober.go:28] interesting pod/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 09 04:15:34.397511 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:34.397474 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" podUID="1e6c6cc6-c6c0-4550-b972-85ac09af5082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 09 04:15:35.397215 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:35.397170 2566 patch_prober.go:28] interesting pod/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 09 04:15:35.397797 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:35.397748 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" podUID="1e6c6cc6-c6c0-4550-b972-85ac09af5082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 09 04:15:36.398413 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:36.398371 2566 patch_prober.go:28] interesting pod/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 09 04:15:36.399035 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:36.398999 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" podUID="1e6c6cc6-c6c0-4550-b972-85ac09af5082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 09 04:15:37.400254 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:37.400224 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:37.400774 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:37.400531 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:37.401303 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:37.401286 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:45.180214 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:45.180169 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m"] Jul 09 04:15:45.180741 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:45.180449 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" podUID="1e6c6cc6-c6c0-4550-b972-85ac09af5082" containerName="istio-proxy" containerID="cri-o://0f12fc73050e1030daca6db43fe8365cc0c709ea8dbfcf0696703efc672f86c0" gracePeriod=30 Jul 09 04:15:50.339915 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.339879 2566 generic.go:356] "Generic (PLEG): container finished" podID="1e6c6cc6-c6c0-4550-b972-85ac09af5082" containerID="0f12fc73050e1030daca6db43fe8365cc0c709ea8dbfcf0696703efc672f86c0" exitCode=0 Jul 09 04:15:50.340326 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.339956 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" event={"ID":"1e6c6cc6-c6c0-4550-b972-85ac09af5082","Type":"ContainerDied","Data":"0f12fc73050e1030daca6db43fe8365cc0c709ea8dbfcf0696703efc672f86c0"} Jul 09 04:15:50.340326 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.340025 2566 scope.go:117] "RemoveContainer" containerID="e9fe406cc275f1fd8638252fe55a4a5c32573f0cfd510aed7346073c286ad914" Jul 09 04:15:50.454557 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.454472 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:50.536681 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.536617 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-token\") pod \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " Jul 09 04:15:50.536681 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.536674 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-workload-socket\") pod \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " Jul 09 04:15:50.536936 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.536707 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-envoy\") pod \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " Jul 09 04:15:50.536936 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.536729 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w74lt\" (UniqueName: \"kubernetes.io/projected/1e6c6cc6-c6c0-4550-b972-85ac09af5082-kube-api-access-w74lt\") pod \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " Jul 09 04:15:50.536936 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.536750 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-workload-certs\") pod \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " Jul 09 04:15:50.536936 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.536802 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-podinfo\") pod \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " Jul 09 04:15:50.536936 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.536833 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-data\") pod \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " Jul 09 04:15:50.537194 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.536969 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istiod-ca-cert\") pod \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " Jul 09 04:15:50.537194 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.537064 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1e6c6cc6-c6c0-4550-b972-85ac09af5082" (UID: "1e6c6cc6-c6c0-4550-b972-85ac09af5082"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:50.537194 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.537075 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1e6c6cc6-c6c0-4550-b972-85ac09af5082" (UID: "1e6c6cc6-c6c0-4550-b972-85ac09af5082"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:50.537194 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.537098 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-credential-socket\") pod \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\" (UID: \"1e6c6cc6-c6c0-4550-b972-85ac09af5082\") " Jul 09 04:15:50.537386 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.537345 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1e6c6cc6-c6c0-4550-b972-85ac09af5082" (UID: "1e6c6cc6-c6c0-4550-b972-85ac09af5082"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:50.537386 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.537367 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1e6c6cc6-c6c0-4550-b972-85ac09af5082" (UID: "1e6c6cc6-c6c0-4550-b972-85ac09af5082"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:15:50.537502 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.537465 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istiod-ca-cert\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:50.537502 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.537483 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-credential-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:50.537502 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.537494 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-workload-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:50.537641 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.537509 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-workload-certs\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:50.537715 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.537690 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-data" (OuterVolumeSpecName: "istio-data") pod "1e6c6cc6-c6c0-4550-b972-85ac09af5082" (UID: "1e6c6cc6-c6c0-4550-b972-85ac09af5082"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:50.539209 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.539179 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e6c6cc6-c6c0-4550-b972-85ac09af5082-kube-api-access-w74lt" (OuterVolumeSpecName: "kube-api-access-w74lt") pod "1e6c6cc6-c6c0-4550-b972-85ac09af5082" (UID: "1e6c6cc6-c6c0-4550-b972-85ac09af5082"). InnerVolumeSpecName "kube-api-access-w74lt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:15:50.539323 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.539211 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1e6c6cc6-c6c0-4550-b972-85ac09af5082" (UID: "1e6c6cc6-c6c0-4550-b972-85ac09af5082"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:15:50.539392 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.539372 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1e6c6cc6-c6c0-4550-b972-85ac09af5082" (UID: "1e6c6cc6-c6c0-4550-b972-85ac09af5082"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:15:50.539392 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.539380 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-token" (OuterVolumeSpecName: "istio-token") pod "1e6c6cc6-c6c0-4550-b972-85ac09af5082" (UID: "1e6c6cc6-c6c0-4550-b972-85ac09af5082"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:15:50.639052 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.639006 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-token\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:50.639052 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.639045 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-envoy\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:50.639052 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.639055 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w74lt\" (UniqueName: \"kubernetes.io/projected/1e6c6cc6-c6c0-4550-b972-85ac09af5082-kube-api-access-w74lt\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:50.639052 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.639065 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-podinfo\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:50.639349 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:50.639074 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e6c6cc6-c6c0-4550-b972-85ac09af5082-istio-data\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:15:51.347493 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:51.347462 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" Jul 09 04:15:51.348010 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:51.347501 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m" event={"ID":"1e6c6cc6-c6c0-4550-b972-85ac09af5082","Type":"ContainerDied","Data":"da7a90e6aa4c5db5483a61b7b0995065b7b81a5cd98fdd405900745a53b01d74"} Jul 09 04:15:51.348010 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:51.347558 2566 scope.go:117] "RemoveContainer" containerID="0f12fc73050e1030daca6db43fe8365cc0c709ea8dbfcf0696703efc672f86c0" Jul 09 04:15:51.372697 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:51.372664 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m"] Jul 09 04:15:51.376439 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:51.376414 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f007b3-openshift-default-6c7d476449-qzb9m"] Jul 09 04:15:51.866672 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:51.866616 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e6c6cc6-c6c0-4550-b972-85ac09af5082" path="/var/lib/kubelet/pods/1e6c6cc6-c6c0-4550-b972-85ac09af5082/volumes" Jul 09 04:15:54.055088 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.055054 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx"] Jul 09 04:15:54.055767 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.055749 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43207361-15b0-4055-98ef-4ba22567837e" containerName="istio-proxy" Jul 09 04:15:54.055767 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.055769 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="43207361-15b0-4055-98ef-4ba22567837e" containerName="istio-proxy" Jul 09 04:15:54.055934 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.055787 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerName="istio-proxy" Jul 09 04:15:54.055934 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.055796 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2a61d84-b931-45b8-9f97-3912efb6ce2a" containerName="istio-proxy" Jul 09 04:15:54.055934 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.055807 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e6c6cc6-c6c0-4550-b972-85ac09af5082" containerName="istio-proxy" Jul 09 04:15:54.055934 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.055816 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e6c6cc6-c6c0-4550-b972-85ac09af5082" containerName="istio-proxy" Jul 09 04:15:54.055934 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.055826 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e6c6cc6-c6c0-4550-b972-85ac09af5082" containerName="istio-proxy" Jul 09 04:15:54.055934 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.055836 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e6c6cc6-c6c0-4550-b972-85ac09af5082" containerName="istio-proxy" Jul 09 04:15:54.055934 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.055924 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e6c6cc6-c6c0-4550-b972-85ac09af5082" containerName="istio-proxy" Jul 09 04:15:54.056267 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.055937 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e6c6cc6-c6c0-4550-b972-85ac09af5082" containerName="istio-proxy" Jul 09 04:15:54.058524 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.058501 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.062104 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.061859 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-90e242-openshift-default-dockercfg-7928n\"" Jul 09 04:15:54.073556 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.073510 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx"] Jul 09 04:15:54.170836 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.170799 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-istio-data\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.171049 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.170856 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srvqx\" (UniqueName: \"kubernetes.io/projected/67957f04-4ef0-403c-b4aa-72e930793f90-kube-api-access-srvqx\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.171049 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.170891 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-istio-envoy\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.171049 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.170925 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67957f04-4ef0-403c-b4aa-72e930793f90-istio-token\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.171049 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.170958 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-credential-socket\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.171049 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.170994 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67957f04-4ef0-403c-b4aa-72e930793f90-istio-podinfo\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.171049 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.171035 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-workload-certs\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.171349 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.171090 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-workload-socket\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.171349 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.171168 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67957f04-4ef0-403c-b4aa-72e930793f90-istiod-ca-cert\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.272373 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.272336 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-istio-data\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.272578 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.272386 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srvqx\" (UniqueName: \"kubernetes.io/projected/67957f04-4ef0-403c-b4aa-72e930793f90-kube-api-access-srvqx\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.272578 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.272413 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-istio-envoy\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.272578 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.272445 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67957f04-4ef0-403c-b4aa-72e930793f90-istio-token\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.272578 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.272475 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-credential-socket\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.272578 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.272511 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67957f04-4ef0-403c-b4aa-72e930793f90-istio-podinfo\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.272928 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.272667 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-workload-certs\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.272928 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.272756 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-workload-socket\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.272928 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.272801 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67957f04-4ef0-403c-b4aa-72e930793f90-istiod-ca-cert\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.272928 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.272912 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-credential-socket\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.273140 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.272985 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-workload-certs\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.273140 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.273093 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-workload-socket\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.273493 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.273467 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67957f04-4ef0-403c-b4aa-72e930793f90-istiod-ca-cert\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.273716 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.273689 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-istio-data\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.275005 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.274981 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-istio-envoy\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.275230 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.275213 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67957f04-4ef0-403c-b4aa-72e930793f90-istio-podinfo\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.281108 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.281087 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srvqx\" (UniqueName: \"kubernetes.io/projected/67957f04-4ef0-403c-b4aa-72e930793f90-kube-api-access-srvqx\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.281186 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.281129 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67957f04-4ef0-403c-b4aa-72e930793f90-istio-token\") pod \"e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.373605 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.373500 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:54.522837 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.522810 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx"] Jul 09 04:15:54.524895 ip-10-0-137-230 kubenswrapper[2566]: W0709 04:15:54.524856 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67957f04_4ef0_403c_b4aa_72e930793f90.slice/crio-a9afdf9e313fd4460110854315e45187dcab5f64401125854c72009a0a3ca73b WatchSource:0}: Error finding container a9afdf9e313fd4460110854315e45187dcab5f64401125854c72009a0a3ca73b: Status 404 returned error can't find the container with id a9afdf9e313fd4460110854315e45187dcab5f64401125854c72009a0a3ca73b Jul 09 04:15:54.527122 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.527086 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:15:54.527225 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.527163 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:15:54.527225 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:54.527193 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:15:55.372942 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:55.372887 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" event={"ID":"67957f04-4ef0-403c-b4aa-72e930793f90","Type":"ContainerStarted","Data":"2e973ee6af35f5ff231c4b0a11935bd25f92ccd2e653f6bb16ab1b6e9c02d7d9"} Jul 09 04:15:55.373386 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:55.373062 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" event={"ID":"67957f04-4ef0-403c-b4aa-72e930793f90","Type":"ContainerStarted","Data":"a9afdf9e313fd4460110854315e45187dcab5f64401125854c72009a0a3ca73b"} Jul 09 04:15:55.374155 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:55.374135 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:55.379577 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:55.379553 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:55.394564 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:55.394508 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" podStartSLOduration=1.3944920760000001 podStartE2EDuration="1.394492076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:15:55.392422735 +0000 UTC m=+2026.024082847" watchObservedRunningTime="2026-07-09 04:15:55.394492076 +0000 UTC m=+2026.026152162" Jul 09 04:15:56.380772 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:56.380741 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:15:56.381807 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:15:56.381787 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:16:12.422907 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:12.422860 2566 patch_prober.go:28] interesting pod/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 04:16:12.423312 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:12.422956 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" podUID="67957f04-4ef0-403c-b4aa-72e930793f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 04:16:13.474162 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:13.474121 2566 generic.go:356] "Generic (PLEG): container finished" podID="67957f04-4ef0-403c-b4aa-72e930793f90" containerID="2e973ee6af35f5ff231c4b0a11935bd25f92ccd2e653f6bb16ab1b6e9c02d7d9" exitCode=0 Jul 09 04:16:13.474906 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:13.474204 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" event={"ID":"67957f04-4ef0-403c-b4aa-72e930793f90","Type":"ContainerDied","Data":"2e973ee6af35f5ff231c4b0a11935bd25f92ccd2e653f6bb16ab1b6e9c02d7d9"} Jul 09 04:16:13.474906 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:13.474785 2566 scope.go:117] "RemoveContainer" containerID="2e973ee6af35f5ff231c4b0a11935bd25f92ccd2e653f6bb16ab1b6e9c02d7d9" Jul 09 04:16:13.475950 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:13.475918 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:16:13.476054 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:13.476015 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:16:13.476118 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:13.476071 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:16:14.486616 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:14.486582 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" event={"ID":"67957f04-4ef0-403c-b4aa-72e930793f90","Type":"ContainerStarted","Data":"7dd8f90aa62c1b6f56edd8b03fd95146059306950bbfbaa3de6104b6dea47540"} Jul 09 04:16:15.374771 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:15.374300 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:16:15.374771 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:15.374439 2566 patch_prober.go:28] interesting pod/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 09 04:16:15.374771 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:15.374486 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" podUID="67957f04-4ef0-403c-b4aa-72e930793f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 09 04:16:16.374571 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:16.374521 2566 patch_prober.go:28] interesting pod/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 09 04:16:16.375141 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:16.374596 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" podUID="67957f04-4ef0-403c-b4aa-72e930793f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 09 04:16:17.375298 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:17.375252 2566 patch_prober.go:28] interesting pod/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 09 04:16:17.375853 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:17.375320 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" podUID="67957f04-4ef0-403c-b4aa-72e930793f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 09 04:16:18.374068 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:18.374027 2566 patch_prober.go:28] interesting pod/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 09 04:16:18.374277 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:18.374097 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" podUID="67957f04-4ef0-403c-b4aa-72e930793f90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 09 04:16:19.377577 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:19.377537 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:16:19.378026 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:19.377779 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:16:19.378429 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:16:19.378410 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:17:09.996308 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:17:09.996270 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 04:17:10.001913 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:17:10.001887 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 04:17:10.002434 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:17:10.002414 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 04:17:10.007371 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:17:10.007343 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 04:19:19.901984 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:19.901947 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx"] Jul 09 04:19:19.902512 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:19.902206 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" podUID="67957f04-4ef0-403c-b4aa-72e930793f90" containerName="istio-proxy" containerID="cri-o://7dd8f90aa62c1b6f56edd8b03fd95146059306950bbfbaa3de6104b6dea47540" gracePeriod=30 Jul 09 04:19:25.170976 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.170906 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:19:25.319771 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.319740 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-credential-socket\") pod \"67957f04-4ef0-403c-b4aa-72e930793f90\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " Jul 09 04:19:25.319947 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.319787 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-workload-certs\") pod \"67957f04-4ef0-403c-b4aa-72e930793f90\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " Jul 09 04:19:25.319947 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.319820 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67957f04-4ef0-403c-b4aa-72e930793f90-istiod-ca-cert\") pod \"67957f04-4ef0-403c-b4aa-72e930793f90\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " Jul 09 04:19:25.319947 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.319841 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67957f04-4ef0-403c-b4aa-72e930793f90-istio-token\") pod \"67957f04-4ef0-403c-b4aa-72e930793f90\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " Jul 09 04:19:25.319947 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.319880 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67957f04-4ef0-403c-b4aa-72e930793f90-istio-podinfo\") pod \"67957f04-4ef0-403c-b4aa-72e930793f90\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " Jul 09 04:19:25.319947 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.319908 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-workload-socket\") pod \"67957f04-4ef0-403c-b4aa-72e930793f90\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " Jul 09 04:19:25.320202 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.319955 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srvqx\" (UniqueName: \"kubernetes.io/projected/67957f04-4ef0-403c-b4aa-72e930793f90-kube-api-access-srvqx\") pod \"67957f04-4ef0-403c-b4aa-72e930793f90\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " Jul 09 04:19:25.320202 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.319989 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-istio-data\") pod \"67957f04-4ef0-403c-b4aa-72e930793f90\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " Jul 09 04:19:25.320202 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.320018 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-istio-envoy\") pod \"67957f04-4ef0-403c-b4aa-72e930793f90\" (UID: \"67957f04-4ef0-403c-b4aa-72e930793f90\") " Jul 09 04:19:25.320202 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.320076 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "67957f04-4ef0-403c-b4aa-72e930793f90" (UID: "67957f04-4ef0-403c-b4aa-72e930793f90"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:19:25.320202 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.320089 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "67957f04-4ef0-403c-b4aa-72e930793f90" (UID: "67957f04-4ef0-403c-b4aa-72e930793f90"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:19:25.320525 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.320294 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "67957f04-4ef0-403c-b4aa-72e930793f90" (UID: "67957f04-4ef0-403c-b4aa-72e930793f90"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:19:25.320525 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.320321 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67957f04-4ef0-403c-b4aa-72e930793f90-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "67957f04-4ef0-403c-b4aa-72e930793f90" (UID: "67957f04-4ef0-403c-b4aa-72e930793f90"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:19:25.320525 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.320460 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-workload-certs\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:19:25.320525 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.320479 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67957f04-4ef0-403c-b4aa-72e930793f90-istiod-ca-cert\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:19:25.320525 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.320496 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-workload-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:19:25.320525 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.320510 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-credential-socket\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:19:25.321184 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.321107 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-istio-data" (OuterVolumeSpecName: "istio-data") pod "67957f04-4ef0-403c-b4aa-72e930793f90" (UID: "67957f04-4ef0-403c-b4aa-72e930793f90"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:19:25.322293 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.322257 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67957f04-4ef0-403c-b4aa-72e930793f90-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "67957f04-4ef0-403c-b4aa-72e930793f90" (UID: "67957f04-4ef0-403c-b4aa-72e930793f90"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:19:25.322388 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.322329 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "67957f04-4ef0-403c-b4aa-72e930793f90" (UID: "67957f04-4ef0-403c-b4aa-72e930793f90"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:19:25.322592 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.322566 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67957f04-4ef0-403c-b4aa-72e930793f90-kube-api-access-srvqx" (OuterVolumeSpecName: "kube-api-access-srvqx") pod "67957f04-4ef0-403c-b4aa-72e930793f90" (UID: "67957f04-4ef0-403c-b4aa-72e930793f90"). InnerVolumeSpecName "kube-api-access-srvqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:19:25.322592 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.322581 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67957f04-4ef0-403c-b4aa-72e930793f90-istio-token" (OuterVolumeSpecName: "istio-token") pod "67957f04-4ef0-403c-b4aa-72e930793f90" (UID: "67957f04-4ef0-403c-b4aa-72e930793f90"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:19:25.421900 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.421808 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-istio-data\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:19:25.421900 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.421838 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67957f04-4ef0-403c-b4aa-72e930793f90-istio-envoy\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:19:25.421900 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.421847 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67957f04-4ef0-403c-b4aa-72e930793f90-istio-token\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:19:25.421900 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.421858 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67957f04-4ef0-403c-b4aa-72e930793f90-istio-podinfo\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:19:25.421900 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.421867 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srvqx\" (UniqueName: \"kubernetes.io/projected/67957f04-4ef0-403c-b4aa-72e930793f90-kube-api-access-srvqx\") on node \"ip-10-0-137-230.ec2.internal\" DevicePath \"\"" Jul 09 04:19:25.488948 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.488910 2566 generic.go:356] "Generic (PLEG): container finished" podID="67957f04-4ef0-403c-b4aa-72e930793f90" containerID="7dd8f90aa62c1b6f56edd8b03fd95146059306950bbfbaa3de6104b6dea47540" exitCode=0 Jul 09 04:19:25.489121 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.488972 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" Jul 09 04:19:25.489121 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.488992 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" event={"ID":"67957f04-4ef0-403c-b4aa-72e930793f90","Type":"ContainerDied","Data":"7dd8f90aa62c1b6f56edd8b03fd95146059306950bbfbaa3de6104b6dea47540"} Jul 09 04:19:25.489121 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.489048 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx" event={"ID":"67957f04-4ef0-403c-b4aa-72e930793f90","Type":"ContainerDied","Data":"a9afdf9e313fd4460110854315e45187dcab5f64401125854c72009a0a3ca73b"} Jul 09 04:19:25.489121 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.489071 2566 scope.go:117] "RemoveContainer" containerID="7dd8f90aa62c1b6f56edd8b03fd95146059306950bbfbaa3de6104b6dea47540" Jul 09 04:19:25.498721 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.498701 2566 scope.go:117] "RemoveContainer" containerID="2e973ee6af35f5ff231c4b0a11935bd25f92ccd2e653f6bb16ab1b6e9c02d7d9" Jul 09 04:19:25.507568 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.507545 2566 scope.go:117] "RemoveContainer" containerID="7dd8f90aa62c1b6f56edd8b03fd95146059306950bbfbaa3de6104b6dea47540" Jul 09 04:19:25.507890 ip-10-0-137-230 kubenswrapper[2566]: E0709 04:19:25.507867 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dd8f90aa62c1b6f56edd8b03fd95146059306950bbfbaa3de6104b6dea47540\": container with ID starting with 7dd8f90aa62c1b6f56edd8b03fd95146059306950bbfbaa3de6104b6dea47540 not found: ID does not exist" containerID="7dd8f90aa62c1b6f56edd8b03fd95146059306950bbfbaa3de6104b6dea47540" Jul 09 04:19:25.507949 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.507901 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dd8f90aa62c1b6f56edd8b03fd95146059306950bbfbaa3de6104b6dea47540"} err="failed to get container status \"7dd8f90aa62c1b6f56edd8b03fd95146059306950bbfbaa3de6104b6dea47540\": rpc error: code = NotFound desc = could not find container \"7dd8f90aa62c1b6f56edd8b03fd95146059306950bbfbaa3de6104b6dea47540\": container with ID starting with 7dd8f90aa62c1b6f56edd8b03fd95146059306950bbfbaa3de6104b6dea47540 not found: ID does not exist" Jul 09 04:19:25.507949 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.507920 2566 scope.go:117] "RemoveContainer" containerID="2e973ee6af35f5ff231c4b0a11935bd25f92ccd2e653f6bb16ab1b6e9c02d7d9" Jul 09 04:19:25.508159 ip-10-0-137-230 kubenswrapper[2566]: E0709 04:19:25.508138 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e973ee6af35f5ff231c4b0a11935bd25f92ccd2e653f6bb16ab1b6e9c02d7d9\": container with ID starting with 2e973ee6af35f5ff231c4b0a11935bd25f92ccd2e653f6bb16ab1b6e9c02d7d9 not found: ID does not exist" containerID="2e973ee6af35f5ff231c4b0a11935bd25f92ccd2e653f6bb16ab1b6e9c02d7d9" Jul 09 04:19:25.508221 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.508169 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e973ee6af35f5ff231c4b0a11935bd25f92ccd2e653f6bb16ab1b6e9c02d7d9"} err="failed to get container status \"2e973ee6af35f5ff231c4b0a11935bd25f92ccd2e653f6bb16ab1b6e9c02d7d9\": rpc error: code = NotFound desc = could not find container \"2e973ee6af35f5ff231c4b0a11935bd25f92ccd2e653f6bb16ab1b6e9c02d7d9\": container with ID starting with 2e973ee6af35f5ff231c4b0a11935bd25f92ccd2e653f6bb16ab1b6e9c02d7d9 not found: ID does not exist" Jul 09 04:19:25.511776 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.511751 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx"] Jul 09 04:19:25.517212 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.517189 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-90e242-openshift-default-7787b7c547-stvgx"] Jul 09 04:19:25.526559 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.526536 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lljdd_2a00fb81-09f2-4b20-b6e4-cea0d989aaf1/manager/0.log" Jul 09 04:19:25.650184 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.650150 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-869986db6b-kzj9d_71bc4735-8713-43ac-8d85-4e4cbf21a181/manager/0.log" Jul 09 04:19:25.766012 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.765906 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4trxg_8ee4103b-2e52-4965-ada7-cbe517269855/server/0.log" Jul 09 04:19:25.866212 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.866176 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67957f04-4ef0-403c-b4aa-72e930793f90" path="/var/lib/kubelet/pods/67957f04-4ef0-403c-b4aa-72e930793f90/volumes" Jul 09 04:19:25.888270 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:25.888240 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-x2tlh_b07779a1-f238-4a74-89d5-2af5a916fac8/manager/0.log" Jul 09 04:19:26.270485 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:26.270452 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645c4d5897-sgf4n_e15e6c75-3643-4caa-b658-6e7c30537173/copy-manifests/0.log" Jul 09 04:19:26.276966 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:26.276942 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645c4d5897-sgf4n_e15e6c75-3643-4caa-b658-6e7c30537173/manager/0.log" Jul 09 04:19:27.728369 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:27.728336 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-zqkrv_f3ca1044-684a-4e0d-ad25-3de28402acf8/manager/0.log" Jul 09 04:19:27.848114 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:27.848087 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-tbh9b_64809dbb-f2d7-4396-b31b-29631d95411b/manager/0.log" Jul 09 04:19:28.083455 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:28.083367 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-zp778_0468c160-fd4b-4bdc-8be4-f21218583524/registry-server/0.log" Jul 09 04:19:28.469238 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:28.469198 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-wt664_a7bb340b-197e-4341-bf20-5f47299cb3fa/manager/0.log" Jul 09 04:19:28.839076 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:28.838969 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678fm686_5070aff3-9714-49c1-abb4-b5b320d8f0b7/istio-proxy/0.log" Jul 09 04:19:29.082584 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:29.082555 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5796d489b5-d9v5b_ea8d949d-c7f5-43c5-8d70-7259a73dec30/kube-auth-proxy/0.log" Jul 09 04:19:29.307936 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:29.307894 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fw4vn_4953493d-d4b5-4068-b41c-488d1aec4e79/istio-proxy/1.log" Jul 09 04:19:29.544147 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:29.544109 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-jbk6d_c084818f-bc07-4c5e-a79c-f7b44e240330/payload-processing/0.log" Jul 09 04:19:29.666090 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:29.666055 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7dfb6bc4d7-d7vz4_93f8e14e-aea8-496e-b728-1554938d338a/router/0.log" Jul 09 04:19:30.122567 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:30.122476 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-pfdm7_26aeea0c-57f2-48c6-98ac-358c102975b1/main/0.log" Jul 09 04:19:30.367074 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:30.367040 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx4cdx_77f86a35-459b-4416-96d7-700ead2442c5/main/0.log" Jul 09 04:19:37.860272 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:37.860194 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k9f2k_69b8b6fa-1fda-4c62-81d1-c9b27231a274/global-pull-secret-syncer/0.log" Jul 09 04:19:37.999836 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:37.999806 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nfr48_175280b2-1ecd-4cc1-a96a-5a9a13207aef/konnectivity-agent/0.log" Jul 09 04:19:38.075847 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:38.075817 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-230.ec2.internal_e59dcb1db041284efb34c27cebcb17d3/haproxy/0.log" Jul 09 04:19:43.746641 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:43.746612 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-zqkrv_f3ca1044-684a-4e0d-ad25-3de28402acf8/manager/0.log" Jul 09 04:19:43.769372 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:43.769344 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-tbh9b_64809dbb-f2d7-4396-b31b-29631d95411b/manager/0.log" Jul 09 04:19:43.829428 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:43.829398 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-zp778_0468c160-fd4b-4bdc-8be4-f21218583524/registry-server/0.log" Jul 09 04:19:44.102610 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:44.102526 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-wt664_a7bb340b-197e-4341-bf20-5f47299cb3fa/manager/0.log" Jul 09 04:19:45.926117 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:45.926086 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/1.log" Jul 09 04:19:46.003604 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:46.003568 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tl5sq_d01b29b9-9680-4c59-acad-460f17b275f4/cluster-monitoring-operator/0.log" Jul 09 04:19:46.166094 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:46.166068 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ghz69_5cb7fa39-6357-41a1-be15-fb41ca654e1c/node-exporter/0.log" Jul 09 04:19:46.186356 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:46.186286 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ghz69_5cb7fa39-6357-41a1-be15-fb41ca654e1c/kube-rbac-proxy/0.log" Jul 09 04:19:46.207425 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:46.207398 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ghz69_5cb7fa39-6357-41a1-be15-fb41ca654e1c/init-textfile/0.log" Jul 09 04:19:48.173372 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.173346 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-xpkvc_c76314eb-b801-418f-9800-c7b0f42c0f97/networking-console-plugin/0.log" Jul 09 04:19:48.183416 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.183387 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5"] Jul 09 04:19:48.183997 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.183977 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67957f04-4ef0-403c-b4aa-72e930793f90" containerName="istio-proxy" Jul 09 04:19:48.183997 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.183995 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="67957f04-4ef0-403c-b4aa-72e930793f90" containerName="istio-proxy" Jul 09 04:19:48.184129 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.184006 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67957f04-4ef0-403c-b4aa-72e930793f90" containerName="istio-proxy" Jul 09 04:19:48.184129 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.184011 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="67957f04-4ef0-403c-b4aa-72e930793f90" containerName="istio-proxy" Jul 09 04:19:48.184129 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.184084 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="67957f04-4ef0-403c-b4aa-72e930793f90" containerName="istio-proxy" Jul 09 04:19:48.187205 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.187191 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.189874 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.189857 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b9skj\"/\"openshift-service-ca.crt\"" Jul 09 04:19:48.191152 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.191133 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b9skj\"/\"default-dockercfg-2zmlk\"" Jul 09 04:19:48.191241 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.191141 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b9skj\"/\"kube-root-ca.crt\"" Jul 09 04:19:48.195574 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.195554 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5"] Jul 09 04:19:48.211197 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.211166 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a59148ce-3c61-409c-9535-a06fa2d48ce2-podres\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.211452 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.211430 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a59148ce-3c61-409c-9535-a06fa2d48ce2-sys\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.211605 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.211591 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a59148ce-3c61-409c-9535-a06fa2d48ce2-proc\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.211802 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.211785 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsttc\" (UniqueName: \"kubernetes.io/projected/a59148ce-3c61-409c-9535-a06fa2d48ce2-kube-api-access-nsttc\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.211933 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.211920 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a59148ce-3c61-409c-9535-a06fa2d48ce2-lib-modules\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.313224 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.313181 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a59148ce-3c61-409c-9535-a06fa2d48ce2-sys\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.313431 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.313238 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a59148ce-3c61-409c-9535-a06fa2d48ce2-proc\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.313431 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.313278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsttc\" (UniqueName: \"kubernetes.io/projected/a59148ce-3c61-409c-9535-a06fa2d48ce2-kube-api-access-nsttc\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.313431 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.313311 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a59148ce-3c61-409c-9535-a06fa2d48ce2-sys\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.313431 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.313315 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a59148ce-3c61-409c-9535-a06fa2d48ce2-lib-modules\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.313431 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.313389 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a59148ce-3c61-409c-9535-a06fa2d48ce2-proc\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.313628 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.313493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a59148ce-3c61-409c-9535-a06fa2d48ce2-podres\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.313628 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.313499 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a59148ce-3c61-409c-9535-a06fa2d48ce2-lib-modules\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.313628 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.313619 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a59148ce-3c61-409c-9535-a06fa2d48ce2-podres\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.321556 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.321533 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsttc\" (UniqueName: \"kubernetes.io/projected/a59148ce-3c61-409c-9535-a06fa2d48ce2-kube-api-access-nsttc\") pod \"perf-node-gather-daemonset-cpdq5\" (UID: \"a59148ce-3c61-409c-9535-a06fa2d48ce2\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.498580 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.498477 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:48.634628 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.634602 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5"] Jul 09 04:19:48.636469 ip-10-0-137-230 kubenswrapper[2566]: W0709 04:19:48.636443 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda59148ce_3c61_409c_9535_a06fa2d48ce2.slice/crio-5ac75690d27465776ec97c200402359980dfc0ab619b78aaee8fb079c1970bdb WatchSource:0}: Error finding container 5ac75690d27465776ec97c200402359980dfc0ab619b78aaee8fb079c1970bdb: Status 404 returned error can't find the container with id 5ac75690d27465776ec97c200402359980dfc0ab619b78aaee8fb079c1970bdb Jul 09 04:19:48.869255 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.869224 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/1.log" Jul 09 04:19:48.873491 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:48.873472 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gskrv_d7993763-77e3-47b3-a32d-aa4b7c406413/console-operator/2.log" Jul 09 04:19:49.351168 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:49.351138 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-tkp5v_9fb07406-da35-4f28-a95c-558d2345c847/download-server/0.log" Jul 09 04:19:49.618762 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:49.618671 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" event={"ID":"a59148ce-3c61-409c-9535-a06fa2d48ce2","Type":"ContainerStarted","Data":"8b0912d8f87a28ceb391f2e4bd04d2920b19197c97285ad8d6c4b034de8dc67c"} Jul 09 04:19:49.618762 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:49.618724 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" event={"ID":"a59148ce-3c61-409c-9535-a06fa2d48ce2","Type":"ContainerStarted","Data":"5ac75690d27465776ec97c200402359980dfc0ab619b78aaee8fb079c1970bdb"} Jul 09 04:19:49.618762 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:49.618753 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:49.635349 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:49.635277 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" podStartSLOduration=1.635262877 podStartE2EDuration="1.635262877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:19:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:19:49.63464041 +0000 UTC m=+2260.266300497" watchObservedRunningTime="2026-07-09 04:19:49.635262877 +0000 UTC m=+2260.266922963" Jul 09 04:19:49.946696 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:49.946665 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-2tpsp_f8a79e10-841e-4cbc-83f5-565ac6b5dfcc/volume-data-source-validator/0.log" Jul 09 04:19:51.071337 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:51.071264 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mzcvn_64ca5948-6243-4377-9c91-a3da567aff5e/dns/0.log" Jul 09 04:19:51.092316 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:51.092292 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mzcvn_64ca5948-6243-4377-9c91-a3da567aff5e/kube-rbac-proxy/0.log" Jul 09 04:19:51.139959 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:51.139928 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mf9s5_6ac5e518-8402-4297-8ce7-4f083d9c2df2/dns-node-resolver/0.log" Jul 09 04:19:51.709510 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:51.709483 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6dggs_6da3145c-5f91-44ca-8465-ce298a4ee12e/node-ca/0.log" Jul 09 04:19:52.731032 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:52.731000 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678fm686_5070aff3-9714-49c1-abb4-b5b320d8f0b7/istio-proxy/0.log" Jul 09 04:19:52.929574 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:52.929549 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5796d489b5-d9v5b_ea8d949d-c7f5-43c5-8d70-7259a73dec30/kube-auth-proxy/0.log" Jul 09 04:19:52.994993 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:52.994922 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fw4vn_4953493d-d4b5-4068-b41c-488d1aec4e79/istio-proxy/0.log" Jul 09 04:19:53.023490 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:53.023458 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fw4vn_4953493d-d4b5-4068-b41c-488d1aec4e79/istio-proxy/1.log" Jul 09 04:19:53.110158 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:53.110131 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-jbk6d_c084818f-bc07-4c5e-a79c-f7b44e240330/payload-processing/0.log" Jul 09 04:19:53.132482 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:53.132450 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7dfb6bc4d7-d7vz4_93f8e14e-aea8-496e-b728-1554938d338a/router/0.log" Jul 09 04:19:53.855055 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:53.855026 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b64b4_5e6ba6a4-190b-4e10-888b-b9f0ed2f0a87/serve-healthcheck-canary/0.log" Jul 09 04:19:54.410317 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:54.410283 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-ddhcf_71b0a743-fa60-46e6-9fde-965caa5bdf86/insights-operator/0.log" Jul 09 04:19:54.411041 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:54.411022 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-ddhcf_71b0a743-fa60-46e6-9fde-965caa5bdf86/insights-operator/1.log" Jul 09 04:19:54.434791 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:54.434764 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-55v58_210d8dd5-d21a-4349-92ee-2f55b6a0786c/kube-rbac-proxy/0.log" Jul 09 04:19:54.460280 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:54.460253 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-55v58_210d8dd5-d21a-4349-92ee-2f55b6a0786c/exporter/0.log" Jul 09 04:19:54.486694 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:54.486641 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-55v58_210d8dd5-d21a-4349-92ee-2f55b6a0786c/extractor/0.log" Jul 09 04:19:55.634494 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:55.634469 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-cpdq5" Jul 09 04:19:56.988508 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:56.988481 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lljdd_2a00fb81-09f2-4b20-b6e4-cea0d989aaf1/manager/0.log" Jul 09 04:19:57.072847 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:57.072815 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-869986db6b-kzj9d_71bc4735-8713-43ac-8d85-4e4cbf21a181/manager/0.log" Jul 09 04:19:57.093686 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:57.093634 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4trxg_8ee4103b-2e52-4965-ada7-cbe517269855/server/0.log" Jul 09 04:19:57.157273 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:57.157243 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-x2tlh_b07779a1-f238-4a74-89d5-2af5a916fac8/manager/0.log" Jul 09 04:19:57.393765 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:57.393737 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645c4d5897-sgf4n_e15e6c75-3643-4caa-b658-6e7c30537173/manager/0.log" Jul 09 04:19:57.414675 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:57.414571 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645c4d5897-sgf4n_e15e6c75-3643-4caa-b658-6e7c30537173/copy-manifests/0.log" Jul 09 04:19:59.073509 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:59.073473 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678fm686_5070aff3-9714-49c1-abb4-b5b320d8f0b7/istio-proxy/0.log" Jul 09 04:19:59.255943 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:59.255917 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5796d489b5-d9v5b_ea8d949d-c7f5-43c5-8d70-7259a73dec30/kube-auth-proxy/0.log" Jul 09 04:19:59.292066 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:59.292016 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fw4vn_4953493d-d4b5-4068-b41c-488d1aec4e79/istio-proxy/0.log" Jul 09 04:19:59.320494 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:59.320461 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fw4vn_4953493d-d4b5-4068-b41c-488d1aec4e79/istio-proxy/1.log" Jul 09 04:19:59.387003 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:59.386928 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-jbk6d_c084818f-bc07-4c5e-a79c-f7b44e240330/payload-processing/0.log" Jul 09 04:19:59.409866 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:19:59.409840 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7dfb6bc4d7-d7vz4_93f8e14e-aea8-496e-b728-1554938d338a/router/0.log" Jul 09 04:20:00.601804 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:00.601775 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678fm686_5070aff3-9714-49c1-abb4-b5b320d8f0b7/istio-proxy/0.log" Jul 09 04:20:00.810231 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:00.810190 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5796d489b5-d9v5b_ea8d949d-c7f5-43c5-8d70-7259a73dec30/kube-auth-proxy/0.log" Jul 09 04:20:00.848003 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:00.847970 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fw4vn_4953493d-d4b5-4068-b41c-488d1aec4e79/istio-proxy/0.log" Jul 09 04:20:00.876346 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:00.876261 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fw4vn_4953493d-d4b5-4068-b41c-488d1aec4e79/istio-proxy/1.log" Jul 09 04:20:00.951357 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:00.951326 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-jbk6d_c084818f-bc07-4c5e-a79c-f7b44e240330/payload-processing/0.log" Jul 09 04:20:00.967174 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:00.967152 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7dfb6bc4d7-d7vz4_93f8e14e-aea8-496e-b728-1554938d338a/router/0.log" Jul 09 04:20:04.037535 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:04.037500 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-f842c_898224dd-88b9-418a-9f84-7cc28899c820/migrator/0.log" Jul 09 04:20:04.057970 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:04.057944 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-f842c_898224dd-88b9-418a-9f84-7cc28899c820/graceful-termination/0.log" Jul 09 04:20:04.426717 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:04.426683 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-tmxz8_f2c564d7-7702-4488-8faa-a7c27f1608c0/kube-storage-version-migrator-operator/1.log" Jul 09 04:20:04.427638 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:04.427619 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-tmxz8_f2c564d7-7702-4488-8faa-a7c27f1608c0/kube-storage-version-migrator-operator/0.log" Jul 09 04:20:05.593108 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:05.593083 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tkpv2_e0abbca0-fa5c-40f8-be6d-225daabb1d4e/kube-multus-additional-cni-plugins/0.log" Jul 09 04:20:05.617768 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:05.617737 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tkpv2_e0abbca0-fa5c-40f8-be6d-225daabb1d4e/egress-router-binary-copy/0.log" Jul 09 04:20:05.640705 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:05.640686 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tkpv2_e0abbca0-fa5c-40f8-be6d-225daabb1d4e/cni-plugins/0.log" Jul 09 04:20:05.663049 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:05.663021 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tkpv2_e0abbca0-fa5c-40f8-be6d-225daabb1d4e/bond-cni-plugin/0.log" Jul 09 04:20:05.687270 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:05.687246 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tkpv2_e0abbca0-fa5c-40f8-be6d-225daabb1d4e/routeoverride-cni/0.log" Jul 09 04:20:05.715694 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:05.715670 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tkpv2_e0abbca0-fa5c-40f8-be6d-225daabb1d4e/whereabouts-cni-bincopy/0.log" Jul 09 04:20:05.743981 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:05.743957 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tkpv2_e0abbca0-fa5c-40f8-be6d-225daabb1d4e/whereabouts-cni/0.log" Jul 09 04:20:05.956033 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:05.956003 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nk6tf_167f4240-06e8-45a9-8770-56c395498f7d/kube-multus/0.log" Jul 09 04:20:06.056816 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:06.056790 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r9c5k_c275241c-186f-492a-952a-cb6ed44c560f/network-metrics-daemon/0.log" Jul 09 04:20:06.078274 ip-10-0-137-230 kubenswrapper[2566]: I0709 04:20:06.078243 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r9c5k_c275241c-186f-492a-952a-cb6ed44c560f/kube-rbac-proxy/0.log"