Jul 29 00:34:06.786884 ip-10-0-135-11 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 29 00:34:06.786896 ip-10-0-135-11 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 29 00:34:06.786906 ip-10-0-135-11 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 29 00:34:06.787348 ip-10-0-135-11 systemd[1]: Failed to start Kubernetes Kubelet. Jul 29 00:34:16.799827 ip-10-0-135-11 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 29 00:34:16.799845 ip-10-0-135-11 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 94d2cb815b724accbe53e4cde1544a09 -- Jul 29 00:35:48.228616 ip-10-0-135-11 systemd[1]: Starting Kubernetes Kubelet... Jul 29 00:35:48.612251 ip-10-0-135-11 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 00:35:48.612251 ip-10-0-135-11 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 29 00:35:48.612251 ip-10-0-135-11 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 00:35:48.612251 ip-10-0-135-11 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 29 00:35:48.612251 ip-10-0-135-11 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 00:35:48.613064 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.612986 2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 29 00:35:48.619860 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619843 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 00:35:48.619860 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619857 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 00:35:48.619860 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619861 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619865 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619868 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619871 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619874 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619876 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619880 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619882 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619885 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619887 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619890 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619893 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619896 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619898 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619901 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619904 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619907 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619909 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619912 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619914 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 00:35:48.619949 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619917 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619920 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619922 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619925 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619927 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619930 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619933 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619935 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619937 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619940 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619942 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619945 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619947 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619950 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619955 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619960 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619963 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619966 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619969 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 00:35:48.620444 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619972 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619974 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619977 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619980 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619982 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619985 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619987 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619990 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619992 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619995 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.619998 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620000 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620004 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620009 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620012 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620014 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620017 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620021 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620023 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 00:35:48.620914 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620026 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620028 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620031 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620034 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620036 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620039 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620042 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620045 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620048 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620050 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620053 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620059 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620062 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620064 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620067 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620069 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620073 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620075 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620077 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 00:35:48.621385 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620080 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620083 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620085 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620088 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620090 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620093 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620095 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620098 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620100 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620102 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620105 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620109 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620112 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620115 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620117 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620120 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620122 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620125 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620127 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620130 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 29 00:35:48.621847 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620855 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620863 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620867 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620869 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620872 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620875 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620878 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620881 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620885 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620888 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620890 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620893 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620895 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620898 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620901 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620903 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620906 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620908 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620911 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620914 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 00:35:48.622410 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620916 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620919 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620922 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620925 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620928 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620930 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620933 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620935 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620940 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620944 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620947 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620950 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620953 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620956 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620959 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620962 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620965 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620967 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620971 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 00:35:48.622905 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620974 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620976 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620979 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620982 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620984 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620987 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620989 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620992 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620994 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620997 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.620999 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621002 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621004 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621007 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621009 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621012 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621014 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621017 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621020 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621022 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 00:35:48.623382 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621025 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621027 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621030 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621032 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621035 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621038 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621042 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621044 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621048 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621051 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621054 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621058 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621061 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621063 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621066 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621068 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621071 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621074 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621076 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 00:35:48.623864 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621079 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621081 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621084 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621087 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621089 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621092 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621095 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621098 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621100 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621103 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621105 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621108 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621110 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621113 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621115 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621118 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621120 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621123 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621127 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 00:35:48.624336 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621130 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621133 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621536 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621541 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621545 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621549 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621552 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621555 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621558 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621561 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621564 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621567 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621570 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621572 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621575 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621580 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621582 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621585 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621588 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621591 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 00:35:48.624834 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621594 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621596 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621598 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621601 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621604 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621607 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621609 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621612 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621614 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621617 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621620 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621622 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621625 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621629 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621631 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621635 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621637 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621640 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621643 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 00:35:48.625318 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621646 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621648 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621651 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621654 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621656 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621659 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621661 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621664 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621668 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621671 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621673 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621676 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621680 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621682 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621685 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621687 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621690 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621692 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621695 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621699 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 00:35:48.625858 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621702 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621705 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621708 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621712 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621715 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621717 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621720 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621722 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621725 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621727 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621730 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621732 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621735 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621738 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621740 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621743 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621745 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621747 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621750 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621753 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 00:35:48.626447 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621756 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621759 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621761 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621764 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621769 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621773 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621776 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621778 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621781 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621784 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621788 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621791 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621794 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621796 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621799 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621802 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621804 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621807 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 00:35:48.626967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621810 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621813 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621815 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.621818 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621879 2558 flags.go:64] FLAG: --address="0.0.0.0" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621890 2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621897 2558 flags.go:64] FLAG: --anonymous-auth="true" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621901 2558 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621906 2558 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621909 2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621913 2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621916 2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621920 2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621923 2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621927 2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621930 2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621933 2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621936 2558 flags.go:64] FLAG: --cgroup-root="" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621942 2558 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621945 2558 flags.go:64] FLAG: --client-ca-file="" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621948 2558 flags.go:64] FLAG: --cloud-provider="external" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621951 2558 flags.go:64] FLAG: --cluster-dns="[]" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621956 2558 flags.go:64] FLAG: --cluster-domain="" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621959 2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 29 00:35:48.627433 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621962 2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621965 2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621968 2558 flags.go:64] FLAG: --container-log-max-files="5" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621972 2558 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621975 2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621978 2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621981 2558 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621984 2558 flags.go:64] FLAG: --contention-profiling="false" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621987 2558 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621990 2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621993 2558 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621995 2558 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.621999 2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622002 2558 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622005 2558 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622008 2558 flags.go:64] FLAG: --enable-load-reader="false" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622011 2558 flags.go:64] FLAG: --enable-server="true" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622014 2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622018 2558 flags.go:64] FLAG: --event-burst="100" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622021 2558 flags.go:64] FLAG: --event-qps="50" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622024 2558 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622026 2558 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622031 2558 flags.go:64] FLAG: --eviction-hard="" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622035 2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 29 00:35:48.628019 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622038 2558 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622041 2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622046 2558 flags.go:64] FLAG: --eviction-soft="" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622048 2558 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622051 2558 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622054 2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622057 2558 flags.go:64] FLAG: --experimental-mounter-path="" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622060 2558 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622063 2558 flags.go:64] FLAG: --fail-swap-on="true" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622066 2558 flags.go:64] FLAG: --feature-gates="" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622069 2558 flags.go:64] FLAG: --file-check-frequency="20s" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622072 2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622075 2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622078 2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622081 2558 flags.go:64] FLAG: --healthz-port="10248" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622084 2558 flags.go:64] FLAG: --help="false" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622087 2558 flags.go:64] FLAG: --hostname-override="ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622090 2558 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622093 2558 flags.go:64] FLAG: --http-check-frequency="20s" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622096 2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622099 2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622103 2558 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622105 2558 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 29 00:35:48.628611 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622108 2558 flags.go:64] FLAG: --image-service-endpoint="" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622111 2558 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622114 2558 flags.go:64] FLAG: --kube-api-burst="100" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622117 2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622120 2558 flags.go:64] FLAG: --kube-api-qps="50" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622123 2558 flags.go:64] FLAG: --kube-reserved="" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622125 2558 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622129 2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622132 2558 flags.go:64] FLAG: --kubelet-cgroups="" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622135 2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622138 2558 flags.go:64] FLAG: --lock-file="" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622141 2558 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622144 2558 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622147 2558 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622152 2558 flags.go:64] FLAG: --log-json-split-stream="false" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622155 2558 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622158 2558 flags.go:64] FLAG: --log-text-split-stream="false" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622161 2558 flags.go:64] FLAG: --logging-format="text" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622164 2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622167 2558 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622170 2558 flags.go:64] FLAG: --manifest-url="" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622172 2558 flags.go:64] FLAG: --manifest-url-header="" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622176 2558 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622179 2558 flags.go:64] FLAG: --max-open-files="1000000" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622183 2558 flags.go:64] FLAG: --max-pods="110" Jul 29 00:35:48.629195 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622186 2558 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622189 2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622192 2558 flags.go:64] FLAG: --memory-manager-policy="None" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622195 2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622198 2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622200 2558 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622203 2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622209 2558 flags.go:64] FLAG: --node-status-max-images="50" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622213 2558 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622216 2558 flags.go:64] FLAG: --oom-score-adj="-999" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622219 2558 flags.go:64] FLAG: --pod-cidr="" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622222 2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622232 2558 flags.go:64] FLAG: --pod-manifest-path="" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622234 2558 flags.go:64] FLAG: --pod-max-pids="-1" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622239 2558 flags.go:64] FLAG: --pods-per-core="0" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622241 2558 flags.go:64] FLAG: --port="10250" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622244 2558 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622247 2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0973014b3f6f1e528" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622250 2558 flags.go:64] FLAG: --qos-reserved="" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622255 2558 flags.go:64] FLAG: --read-only-port="10255" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622258 2558 flags.go:64] FLAG: --register-node="true" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622260 2558 flags.go:64] FLAG: --register-with-taints="" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622268 2558 flags.go:64] FLAG: --registry-burst="10" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622271 2558 flags.go:64] FLAG: --registry-qps="5" Jul 29 00:35:48.629813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622274 2558 flags.go:64] FLAG: --reserved-cpus="" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622277 2558 flags.go:64] FLAG: --reserved-memory="" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622280 2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622283 2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622286 2558 flags.go:64] FLAG: --rotate-certificates="false" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622289 2558 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622291 2558 flags.go:64] FLAG: --runonce="false" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622294 2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622297 2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622300 2558 flags.go:64] FLAG: --seccomp-default="false" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622303 2558 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622321 2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622326 2558 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622330 2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622335 2558 flags.go:64] FLAG: --storage-driver-password="root" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622337 2558 flags.go:64] FLAG: --storage-driver-secure="false" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622340 2558 flags.go:64] FLAG: --storage-driver-table="stats" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622343 2558 flags.go:64] FLAG: --storage-driver-user="root" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622346 2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622349 2558 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622359 2558 flags.go:64] FLAG: --system-cgroups="" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622362 2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622369 2558 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622372 2558 flags.go:64] FLAG: --tls-cert-file="" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622375 2558 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 29 00:35:48.630411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622379 2558 flags.go:64] FLAG: --tls-min-version="" Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622381 2558 flags.go:64] FLAG: --tls-private-key-file="" Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622385 2558 flags.go:64] FLAG: --topology-manager-policy="none" Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622388 2558 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622391 2558 flags.go:64] FLAG: --topology-manager-scope="container" Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622394 2558 flags.go:64] FLAG: --v="2" Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622402 2558 flags.go:64] FLAG: --version="false" Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622406 2558 flags.go:64] FLAG: --vmodule="" Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622409 2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622413 2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622521 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622525 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622529 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622532 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622535 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622537 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622540 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622543 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622546 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622549 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622552 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622554 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 00:35:48.631066 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622557 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622560 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622563 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622565 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622568 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622571 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622574 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622579 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622582 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622584 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622587 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622589 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622593 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622596 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622598 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622601 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622604 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622606 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622609 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622611 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 00:35:48.631606 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622614 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622618 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622621 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622624 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622627 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622630 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622633 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622636 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622638 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622640 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622643 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622645 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622648 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622651 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622653 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622656 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622658 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622661 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622663 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 00:35:48.632077 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622668 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622671 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622673 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622676 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622678 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622682 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622684 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622687 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622689 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622692 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622696 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622699 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622702 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622705 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622708 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622711 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622714 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622716 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622719 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 00:35:48.632552 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622722 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622724 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622727 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622729 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622731 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622734 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622737 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622739 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622741 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622744 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622747 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622750 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622752 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622756 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622759 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622761 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622764 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622767 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622771 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622773 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 00:35:48.633029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622776 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 00:35:48.633527 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622778 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 00:35:48.633527 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622781 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 00:35:48.633527 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622783 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 00:35:48.633527 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622786 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 00:35:48.633527 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622788 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 00:35:48.633527 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622791 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 00:35:48.633527 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622794 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 00:35:48.633527 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.622798 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 00:35:48.633527 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.622806 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 00:35:48.633527 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.629016 2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 29 00:35:48.633527 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.629030 2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 29 00:35:48.633527 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.629047 2558 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 29 00:35:48.633527 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.629053 2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 29 00:35:48.633527 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629107 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629112 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629116 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629118 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629121 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629124 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629128 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629132 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629135 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629138 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629141 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629143 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629147 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629150 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629153 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629156 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629158 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629161 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629164 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 00:35:48.633877 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629166 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629169 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629172 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629175 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629177 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629180 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629182 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629185 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629187 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629190 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629194 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629197 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629200 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629202 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629205 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629208 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629210 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629213 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629215 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629218 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 00:35:48.634351 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629221 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629223 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629226 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629228 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629231 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629233 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629236 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629239 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629241 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629244 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629248 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629251 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629255 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629257 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629260 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629263 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629265 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629268 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629270 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629273 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 00:35:48.634843 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629275 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629278 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629281 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629284 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629286 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629289 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629293 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629296 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629299 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629302 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629318 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629321 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629323 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629326 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629329 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629332 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629334 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629337 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629339 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 00:35:48.635326 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629342 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629345 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629347 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629350 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629352 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629355 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629358 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629361 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629363 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629366 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629368 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629371 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629373 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629376 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629378 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629382 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629385 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629387 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629390 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629393 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 00:35:48.635830 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629395 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 00:35:48.636892 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.629400 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 00:35:48.636892 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629510 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 00:35:48.636892 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629515 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 00:35:48.636892 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629518 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 00:35:48.636892 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629521 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 00:35:48.636892 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629525 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 00:35:48.636892 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629528 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 00:35:48.636892 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629531 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 00:35:48.636892 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629534 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 00:35:48.636892 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629537 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 00:35:48.636892 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629539 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 00:35:48.636892 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629542 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 00:35:48.636892 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629546 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 00:35:48.636892 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629548 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629551 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629553 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629555 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629558 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629560 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629563 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629566 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629568 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629571 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629573 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629575 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629578 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629580 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629584 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629586 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629589 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629592 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629594 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629597 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 00:35:48.637238 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629599 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629602 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629605 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629607 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629610 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629612 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629615 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629617 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629620 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629622 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629625 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629627 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629630 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629633 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629636 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629638 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629640 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629643 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629646 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629648 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 00:35:48.637755 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629651 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629653 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629655 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629658 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629660 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629663 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629666 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629669 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629671 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629674 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629676 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629679 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629681 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629684 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629686 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629689 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629691 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629693 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629696 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 00:35:48.638258 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629698 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629702 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629705 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629708 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629711 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629713 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629716 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629719 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629721 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629724 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629726 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629728 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629731 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629733 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629736 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629738 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629741 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629743 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629746 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 00:35:48.638979 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629750 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 00:35:48.639783 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629753 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 00:35:48.639783 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629755 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 00:35:48.639783 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629758 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 00:35:48.639783 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629761 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 00:35:48.639783 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629763 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 00:35:48.639783 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629766 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 00:35:48.639783 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629768 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 00:35:48.639783 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:48.629771 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 00:35:48.639783 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.629776 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 00:35:48.639783 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.629863 2558 server.go:962] "Client rotation is on, will bootstrap in background" Jul 29 00:35:48.639783 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.631781 2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 29 00:35:48.639783 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.633017 2558 server.go:1019] "Starting client certificate rotation" Jul 29 00:35:48.639783 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.633114 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 00:35:48.639783 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.633147 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 00:35:48.640203 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.635210 2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 00:35:48.640203 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.639914 2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 00:35:48.650569 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.650553 2558 log.go:25] "Validated CRI v1 runtime API" Jul 29 00:35:48.652589 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.652573 2558 log.go:25] "Validated CRI v1 image API" Jul 29 00:35:48.653458 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.653439 2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 29 00:35:48.656474 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.656445 2558 fs.go:135] Filesystem UUIDs: map[59d5beb0-b6f7-4592-bdff-d0d5921f00a2:/dev/nvme0n1p3 6a833890-dc9e-4a6e-a498-d98c8e980180:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 29 00:35:48.656549 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.656474 2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 29 00:35:48.662110 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.662003 2558 manager.go:216] Machine: {Timestamp:2026-07-29 00:35:48.660723482 +0000 UTC m=+0.319949365 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100133 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f53b7a29ea40d9a92dce9095d6344 SystemUUID:ec2f53b7-a29e-a40d-9a92-dce9095d6344 BootID:94d2cb81-5b72-4acc-be53-e4cde1544a09 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6d:ef:b8:f3:13 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6d:ef:b8:f3:13 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:47:76:ad:77:15 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 29 00:35:48.662110 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.662102 2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 29 00:35:48.662243 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.662172 2558 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 29 00:35:48.663393 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.663378 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 00:35:48.664919 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.664896 2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 29 00:35:48.665048 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.664922 2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-11.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 29 00:35:48.665623 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.665613 2558 topology_manager.go:138] "Creating topology manager with none policy" Jul 29 00:35:48.665660 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.665626 2558 container_manager_linux.go:309] "Creating device plugin manager" Jul 29 00:35:48.665660 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.665638 2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 00:35:48.666287 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.666277 2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 00:35:48.667123 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.667113 2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 29 00:35:48.668767 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.668756 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 29 00:35:48.668869 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.668860 2558 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 29 00:35:48.668907 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.668887 2558 kubelet.go:486] "Attempting to sync node with API server" Jul 29 00:35:48.668907 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.668896 2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 29 00:35:48.668977 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.668911 2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 29 00:35:48.668977 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.668919 2558 kubelet.go:398] "Adding apiserver pod source" Jul 29 00:35:48.668977 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.668928 2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 29 00:35:48.669645 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.669628 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 00:35:48.669645 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.669648 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 00:35:48.672791 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.672765 2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 29 00:35:48.674907 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.674890 2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 29 00:35:48.674981 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.674924 2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 29 00:35:48.676992 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.676978 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 29 00:35:48.677062 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.676996 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 29 00:35:48.677062 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.677002 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 29 00:35:48.677062 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.677008 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 29 00:35:48.677062 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.677017 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 29 00:35:48.677062 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.677022 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 29 00:35:48.677062 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.677035 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 29 00:35:48.677062 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.677041 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 29 00:35:48.677062 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.677047 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 29 00:35:48.677062 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.677052 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 29 00:35:48.677062 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.677061 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 29 00:35:48.677062 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.677066 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 29 00:35:48.677989 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.677976 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 29 00:35:48.678025 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.677989 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 29 00:35:48.680679 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.680660 2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-11.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 29 00:35:48.681150 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.681136 2558 server.go:1268] "Started kubelet" Jul 29 00:35:48.681233 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.681207 2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 29 00:35:48.681292 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.681239 2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 29 00:35:48.681352 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.681302 2558 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 29 00:35:48.681512 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.681191 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 29 00:35:48.681512 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.681217 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 29 00:35:48.682004 ip-10-0-135-11 systemd[1]: Started Kubernetes Kubelet. Jul 29 00:35:48.683295 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.683278 2558 server.go:310] "Adding debug handlers to kubelet server" Jul 29 00:35:48.683389 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.683332 2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 29 00:35:48.688699 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.687859 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-11.ec2.internal.18c69b90d6d8d7d5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-11.ec2.internal,UID:ip-10-0-135-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-11.ec2.internal,},FirstTimestamp:2026-07-29 00:35:48.681111509 +0000 UTC m=+0.340337394,LastTimestamp:2026-07-29 00:35:48.681111509 +0000 UTC m=+0.340337394,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-11.ec2.internal,}" Jul 29 00:35:48.689038 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.689022 2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 29 00:35:48.689769 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.689756 2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 29 00:35:48.689820 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.689766 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 29 00:35:48.689868 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.689828 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:48.690356 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.690338 2558 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 29 00:35:48.690356 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.690357 2558 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 29 00:35:48.690497 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.690452 2558 reconstruct.go:97] "Volume reconstruction finished" Jul 29 00:35:48.690497 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.690462 2558 reconciler.go:29] "Reconciler: start to sync state" Jul 29 00:35:48.690590 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.690558 2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 29 00:35:48.690694 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.690677 2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 29 00:35:48.690694 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.690691 2558 factory.go:55] Registering systemd factory Jul 29 00:35:48.690694 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.690698 2558 factory.go:223] Registration of the systemd container factory successfully Jul 29 00:35:48.690893 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.690878 2558 factory.go:153] Registering CRI-O factory Jul 29 00:35:48.690932 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.690898 2558 factory.go:223] Registration of the crio container factory successfully Jul 29 00:35:48.690932 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.690926 2558 factory.go:103] Registering Raw factory Jul 29 00:35:48.691023 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.690939 2558 manager.go:1196] Started watching for new ooms in manager Jul 29 00:35:48.691454 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.691437 2558 manager.go:349] Starting recovery of all containers Jul 29 00:35:48.696844 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.696813 2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 29 00:35:48.696961 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.696937 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 29 00:35:48.702980 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.702956 2558 manager.go:354] Recovery completed Jul 29 00:35:48.708224 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.708211 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 00:35:48.710006 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.709986 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory" Jul 29 00:35:48.710079 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.710025 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 00:35:48.710079 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.710041 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID" Jul 29 00:35:48.710666 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.710648 2558 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 29 00:35:48.710666 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.710666 2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 29 00:35:48.710786 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.710653 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 00:35:48.710786 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.710683 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 29 00:35:48.711366 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.711347 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory" Jul 29 00:35:48.711451 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.711379 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 00:35:48.711451 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.711393 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID" Jul 29 00:35:48.712529 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.712509 2558 policy_none.go:49] "None policy: Start" Jul 29 00:35:48.712529 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.712528 2558 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 29 00:35:48.712643 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.712542 2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 29 00:35:48.712643 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.712499 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-11.ec2.internal.18c69b90d891c0e8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-11.ec2.internal,UID:ip-10-0-135-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-11.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-11.ec2.internal,},FirstTimestamp:2026-07-29 00:35:48.710007016 +0000 UTC m=+0.369232902,LastTimestamp:2026-07-29 00:35:48.710007016 +0000 UTC m=+0.369232902,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-11.ec2.internal,}" Jul 29 00:35:48.713450 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.713436 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2w5qd" Jul 29 00:35:48.713857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.713847 2558 policy_none.go:47] "Start" Jul 29 00:35:48.721646 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.721627 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2w5qd" Jul 29 00:35:48.721799 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.721720 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-11.ec2.internal.18c69b90d892247f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-11.ec2.internal,UID:ip-10-0-135-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-11.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-11.ec2.internal,},FirstTimestamp:2026-07-29 00:35:48.710032511 +0000 UTC m=+0.369258395,LastTimestamp:2026-07-29 00:35:48.710032511 +0000 UTC m=+0.369258395,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-11.ec2.internal,}" Jul 29 00:35:48.753759 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.745621 2558 manager.go:337] "Starting Device Plugin manager" Jul 29 00:35:48.753759 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.745658 2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 29 00:35:48.753759 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.745671 2558 server.go:87] "Starting device plugin registration server" Jul 29 00:35:48.753759 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.745881 2558 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 29 00:35:48.753759 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.745893 2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 29 00:35:48.753759 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.746025 2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 29 00:35:48.753759 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.746086 2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 29 00:35:48.753759 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.746093 2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 29 00:35:48.753759 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.746741 2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 29 00:35:48.753759 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.746787 2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:48.781553 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.781525 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 29 00:35:48.782646 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.782630 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 29 00:35:48.782722 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.782667 2558 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 29 00:35:48.782722 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.782687 2558 kubelet.go:2448] "Starting kubelet main sync loop" Jul 29 00:35:48.782809 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.782724 2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 29 00:35:48.785358 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.785339 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 00:35:48.846590 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.846575 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 00:35:48.848180 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.848166 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory" Jul 29 00:35:48.848246 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.848192 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 00:35:48.848246 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.848201 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID" Jul 29 00:35:48.848246 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.848220 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.861234 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.861219 2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.861299 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.861237 2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-11.ec2.internal\": node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:48.883865 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.883704 2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal"] Jul 29 00:35:48.883944 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.883869 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 00:35:48.884596 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.884582 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory" Jul 29 00:35:48.884682 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.884607 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 00:35:48.884682 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.884616 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID" Jul 29 00:35:48.885498 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.885482 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:48.886279 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.886268 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 00:35:48.886405 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.886393 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.886453 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.886420 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 00:35:48.886957 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.886942 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory" Jul 29 00:35:48.887024 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.886971 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 00:35:48.887024 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.886946 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory" Jul 29 00:35:48.887024 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.886982 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID" Jul 29 00:35:48.887024 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.887001 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 00:35:48.887024 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.887012 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID" Jul 29 00:35:48.888097 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.888084 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.888146 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.888106 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 00:35:48.888744 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.888718 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory" Jul 29 00:35:48.888744 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.888741 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 00:35:48.888862 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.888755 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID" Jul 29 00:35:48.891721 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.891700 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f0d2324c5d97061fc92a237a96dbdcf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal\" (UID: \"1f0d2324c5d97061fc92a237a96dbdcf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.891789 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.891738 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f0d2324c5d97061fc92a237a96dbdcf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal\" (UID: \"1f0d2324c5d97061fc92a237a96dbdcf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.891827 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.891782 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe0818b1e6c45ae97ec4ef99e813669f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-11.ec2.internal\" (UID: \"fe0818b1e6c45ae97ec4ef99e813669f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.917869 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.917853 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-11.ec2.internal\" not found" node="ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.921942 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.921928 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-11.ec2.internal\" not found" node="ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.985841 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:48.985824 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:48.992094 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.992071 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f0d2324c5d97061fc92a237a96dbdcf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal\" (UID: \"1f0d2324c5d97061fc92a237a96dbdcf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.992179 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.992103 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f0d2324c5d97061fc92a237a96dbdcf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal\" (UID: \"1f0d2324c5d97061fc92a237a96dbdcf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.992179 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.992118 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe0818b1e6c45ae97ec4ef99e813669f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-11.ec2.internal\" (UID: \"fe0818b1e6c45ae97ec4ef99e813669f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.992179 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.992142 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe0818b1e6c45ae97ec4ef99e813669f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-11.ec2.internal\" (UID: \"fe0818b1e6c45ae97ec4ef99e813669f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.992179 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.992161 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f0d2324c5d97061fc92a237a96dbdcf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal\" (UID: \"1f0d2324c5d97061fc92a237a96dbdcf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" Jul 29 00:35:48.992346 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:48.992182 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f0d2324c5d97061fc92a237a96dbdcf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal\" (UID: \"1f0d2324c5d97061fc92a237a96dbdcf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" Jul 29 00:35:49.086263 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:49.086238 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:49.187017 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:49.186955 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:49.220453 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:49.220433 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" Jul 29 00:35:49.225353 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:49.225330 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal" Jul 29 00:35:49.288038 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:49.288019 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:49.388497 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:49.388479 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:49.488992 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:49.488946 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:49.589443 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:49.589422 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:49.632858 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:49.632837 2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 29 00:35:49.633243 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:49.632966 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 29 00:35:49.690171 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:49.690149 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:49.699733 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:49.699704 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe0818b1e6c45ae97ec4ef99e813669f.slice/crio-59efb2b89eaa57cf17fd100ea6c7f430c1f0ad8a68b506a92283c8ac031cd607 WatchSource:0}: Error finding container 59efb2b89eaa57cf17fd100ea6c7f430c1f0ad8a68b506a92283c8ac031cd607: Status 404 returned error can't find the container with id 59efb2b89eaa57cf17fd100ea6c7f430c1f0ad8a68b506a92283c8ac031cd607 Jul 29 00:35:49.704646 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:49.704549 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 00:35:49.709650 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:49.709630 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f0d2324c5d97061fc92a237a96dbdcf.slice/crio-700af313827fc380b358fdff72846667fb679ae0807dd94e7a212e4da041277d WatchSource:0}: Error finding container 700af313827fc380b358fdff72846667fb679ae0807dd94e7a212e4da041277d: Status 404 returned error can't find the container with id 700af313827fc380b358fdff72846667fb679ae0807dd94e7a212e4da041277d Jul 29 00:35:49.725316 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:49.725276 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 00:30:49 +0000 UTC" deadline="2028-01-14 18:47:39.56582713 +0000 UTC" Jul 29 00:35:49.725316 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:49.725302 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12834h11m49.84052795s" Jul 29 00:35:49.785471 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:49.785394 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" event={"ID":"1f0d2324c5d97061fc92a237a96dbdcf","Type":"ContainerStarted","Data":"700af313827fc380b358fdff72846667fb679ae0807dd94e7a212e4da041277d"} Jul 29 00:35:49.786286 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:49.786267 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal" event={"ID":"fe0818b1e6c45ae97ec4ef99e813669f","Type":"ContainerStarted","Data":"59efb2b89eaa57cf17fd100ea6c7f430c1f0ad8a68b506a92283c8ac031cd607"} Jul 29 00:35:49.790412 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:49.790398 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:49.803762 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:49.803748 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 00:35:49.805166 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:49.805151 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientMemory" Jul 29 00:35:49.805230 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:49.805193 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 00:35:49.805230 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:49.805208 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeHasSufficientPID" Jul 29 00:35:49.848663 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:49.848642 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 00:35:49.892508 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:49.892478 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:49.997830 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:49.997802 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:50.098441 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:50.098371 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-11.ec2.internal\" not found" Jul 29 00:35:50.156956 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.156933 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 00:35:50.191416 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.191374 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" Jul 29 00:35:50.206235 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.206216 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 00:35:50.208030 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.208013 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal" Jul 29 00:35:50.217184 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.217165 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 00:35:50.228037 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.228016 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 00:35:50.670060 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.670034 2558 apiserver.go:52] "Watching apiserver" Jul 29 00:35:50.679130 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.679100 2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 29 00:35:50.679635 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.679595 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn","openshift-cluster-node-tuning-operator/tuned-fjv69","openshift-dns/node-resolver-vpqgd","openshift-image-registry/node-ca-dffdw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal","openshift-multus/multus-additional-cni-plugins-c527k","openshift-multus/network-metrics-daemon-mgvr7","kube-system/konnectivity-agent-cjfg6","kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal","openshift-multus/multus-w8sxh","openshift-network-diagnostics/network-check-target-2kc2k","openshift-network-operator/iptables-alerter-kwth9","openshift-ovn-kubernetes/ovnkube-node-mtt7s","kube-system/global-pull-secret-syncer-jghtb"] Jul 29 00:35:50.681322 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.681295 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.682692 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.682669 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:35:50.682812 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:50.682746 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:35:50.683475 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.683454 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 29 00:35:50.683788 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.683723 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 29 00:35:50.684417 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.684027 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vpqgd" Jul 29 00:35:50.684417 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.684057 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-q2jkw\"" Jul 29 00:35:50.684417 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.684086 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 29 00:35:50.684417 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.684036 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 29 00:35:50.685240 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.685225 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:50.685405 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:50.685288 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:35:50.685740 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.685709 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-swbgk\"" Jul 29 00:35:50.685899 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.685884 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 29 00:35:50.686167 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.686144 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 29 00:35:50.686519 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.686495 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cjfg6" Jul 29 00:35:50.688369 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.688059 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.688369 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.688198 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 29 00:35:50.688841 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.688807 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 29 00:35:50.689006 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.688988 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xf4fn\"" Jul 29 00:35:50.689810 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.689731 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 29 00:35:50.689810 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.689804 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-24qj5\"" Jul 29 00:35:50.690125 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.690107 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kwth9" Jul 29 00:35:50.690210 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.690126 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 29 00:35:50.690474 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.690426 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 29 00:35:50.691360 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.691156 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.692600 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.692340 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 29 00:35:50.692600 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.692377 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 29 00:35:50.692600 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.692392 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 29 00:35:50.692600 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.692586 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7xh2q\"" Jul 29 00:35:50.692866 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.692644 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.693162 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.693137 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 29 00:35:50.693239 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.693171 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 29 00:35:50.693279 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.693171 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 29 00:35:50.693279 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.693254 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 29 00:35:50.694554 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.694534 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mwpcf\"" Jul 29 00:35:50.694825 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.694800 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 29 00:35:50.694936 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.694853 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.695025 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.694941 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 29 00:35:50.695087 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.695057 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 29 00:35:50.695087 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.695072 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 29 00:35:50.695250 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.695229 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tv86b\"" Jul 29 00:35:50.695948 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.695929 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:35:50.696037 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:50.695997 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:35:50.697448 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.697293 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 29 00:35:50.697570 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.697548 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 29 00:35:50.697717 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.697699 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dffdw" Jul 29 00:35:50.697841 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.697699 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 29 00:35:50.698292 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.698274 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z9npk\"" Jul 29 00:35:50.699871 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.699850 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rc7q8\"" Jul 29 00:35:50.700415 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.700107 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 29 00:35:50.700415 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.700298 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 29 00:35:50.700415 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.700364 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 29 00:35:50.702857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.702187 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e60adc84-fa8b-4053-9334-fc85a835ccf7-env-overrides\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.702857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.702215 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-var-lib-cni-multus\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.702857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.702242 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41c070b5-13d1-47ea-a5e4-50f519e7e647-multus-daemon-config\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.702857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.702277 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-dbus\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:50.702857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.702303 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-systemd-units\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.702857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.702326 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb151e18-2bf5-4098-a956-829f7bbf9e4e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.702857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.702393 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-run-netns\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.702857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.702423 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e60adc84-fa8b-4053-9334-fc85a835ccf7-ovn-node-metrics-cert\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.703698 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.703676 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-os-release\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.703783 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.703713 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-etc-openvswitch\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.703783 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.703751 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e60adc84-fa8b-4053-9334-fc85a835ccf7-ovnkube-config\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.703891 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.703781 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-run-openvswitch\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.703891 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.703803 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-run-ovn\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.703891 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.703829 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj8ps\" (UniqueName: \"kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps\") pod \"network-check-target-2kc2k\" (UID: \"c56793a6-2612-4e39-84fd-5c357c49066a\") " pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:35:50.703891 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.703857 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqqrn\" (UniqueName: \"kubernetes.io/projected/cb151e18-2bf5-4098-a956-829f7bbf9e4e-kube-api-access-cqqrn\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.703891 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.703885 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-var-lib-openvswitch\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.704141 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.703909 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-log-socket\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.704141 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.703934 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e60adc84-fa8b-4053-9334-fc85a835ccf7-ovnkube-script-lib\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.704141 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.703972 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-run-k8s-cni-cncf-io\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.704141 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704014 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-multus-conf-dir\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.704141 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704045 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-kubelet-config\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:50.704141 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704070 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a6daeacc-5a05-4ab1-8966-dd67c27e49d2-iptables-alerter-script\") pod \"iptables-alerter-kwth9\" (UID: \"a6daeacc-5a05-4ab1-8966-dd67c27e49d2\") " pod="openshift-network-operator/iptables-alerter-kwth9" Jul 29 00:35:50.704141 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704111 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6daeacc-5a05-4ab1-8966-dd67c27e49d2-host-slash\") pod \"iptables-alerter-kwth9\" (UID: \"a6daeacc-5a05-4ab1-8966-dd67c27e49d2\") " pod="openshift-network-operator/iptables-alerter-kwth9" Jul 29 00:35:50.704141 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704132 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-kubelet\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.704559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704155 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/20137ae0-172b-4165-9f8f-87bb7df1fd80-hosts-file\") pod \"node-resolver-vpqgd\" (UID: \"20137ae0-172b-4165-9f8f-87bb7df1fd80\") " pod="openshift-dns/node-resolver-vpqgd" Jul 29 00:35:50.704559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704185 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-hostroot\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.704559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704219 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-run-multus-certs\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.704559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704242 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-slash\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.704559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704279 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-cni-bin\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.704559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704309 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20137ae0-172b-4165-9f8f-87bb7df1fd80-tmp-dir\") pod \"node-resolver-vpqgd\" (UID: \"20137ae0-172b-4165-9f8f-87bb7df1fd80\") " pod="openshift-dns/node-resolver-vpqgd" Jul 29 00:35:50.704559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704331 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-cnibin\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.704559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704387 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-multus-socket-dir-parent\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.704559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704413 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-etc-kubernetes\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.704559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704438 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-run-ovn-kubernetes\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.704559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704505 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/703966f4-444e-4b3b-854d-3a7b67259320-agent-certs\") pod \"konnectivity-agent-cjfg6\" (UID: \"703966f4-444e-4b3b-854d-3a7b67259320\") " pod="kube-system/konnectivity-agent-cjfg6" Jul 29 00:35:50.704559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704534 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-var-lib-cni-bin\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.704559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704552 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704574 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj8fk\" (UniqueName: \"kubernetes.io/projected/a6daeacc-5a05-4ab1-8966-dd67c27e49d2-kube-api-access-qj8fk\") pod \"iptables-alerter-kwth9\" (UID: \"a6daeacc-5a05-4ab1-8966-dd67c27e49d2\") " pod="openshift-network-operator/iptables-alerter-kwth9" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704614 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jttd\" (UniqueName: \"kubernetes.io/projected/20137ae0-172b-4165-9f8f-87bb7df1fd80-kube-api-access-2jttd\") pod \"node-resolver-vpqgd\" (UID: \"20137ae0-172b-4165-9f8f-87bb7df1fd80\") " pod="openshift-dns/node-resolver-vpqgd" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704642 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb151e18-2bf5-4098-a956-829f7bbf9e4e-system-cni-dir\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704677 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb151e18-2bf5-4098-a956-829f7bbf9e4e-cnibin\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704747 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-system-cni-dir\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704766 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-run-netns\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704787 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-var-lib-kubelet\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704806 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-node-log\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704851 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb151e18-2bf5-4098-a956-829f7bbf9e4e-os-release\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704878 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-multus-cni-dir\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704936 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-cni-netd\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.704974 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb151e18-2bf5-4098-a956-829f7bbf9e4e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.705014 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb151e18-2bf5-4098-a956-829f7bbf9e4e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.705050 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41c070b5-13d1-47ea-a5e4-50f519e7e647-cni-binary-copy\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.705151 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-run-systemd\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.705228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.705186 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz88p\" (UniqueName: \"kubernetes.io/projected/e60adc84-fa8b-4053-9334-fc85a835ccf7-kube-api-access-jz88p\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.706033 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.705214 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnmkj\" (UniqueName: \"kubernetes.io/projected/41c070b5-13d1-47ea-a5e4-50f519e7e647-kube-api-access-tnmkj\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.706033 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.705253 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.706033 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.705293 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/703966f4-444e-4b3b-854d-3a7b67259320-konnectivity-ca\") pod \"konnectivity-agent-cjfg6\" (UID: \"703966f4-444e-4b3b-854d-3a7b67259320\") " pod="kube-system/konnectivity-agent-cjfg6" Jul 29 00:35:50.706033 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.705349 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb151e18-2bf5-4098-a956-829f7bbf9e4e-cni-binary-copy\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.706033 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.705868 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 00:35:50.734268 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.734244 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-krh6p" Jul 29 00:35:50.748314 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.748290 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-krh6p" Jul 29 00:35:50.791798 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.791773 2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 29 00:35:50.806055 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806028 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb151e18-2bf5-4098-a956-829f7bbf9e4e-os-release\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.806179 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806066 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-multus-cni-dir\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.806179 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806094 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-var-lib-kubelet\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.806179 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806118 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-host\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.806179 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806174 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs\") pod \"network-metrics-daemon-mgvr7\" (UID: \"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c\") " pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:35:50.806443 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806183 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb151e18-2bf5-4098-a956-829f7bbf9e4e-os-release\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.806443 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806199 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-cni-netd\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.806443 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806224 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb151e18-2bf5-4098-a956-829f7bbf9e4e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.806443 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806248 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb151e18-2bf5-4098-a956-829f7bbf9e4e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.806443 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806256 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-cni-netd\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.806443 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806269 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41c070b5-13d1-47ea-a5e4-50f519e7e647-cni-binary-copy\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.806443 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806306 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b5888d04-a838-4a13-ac06-5ef697b5ee59-serviceca\") pod \"node-ca-dffdw\" (UID: \"b5888d04-a838-4a13-ac06-5ef697b5ee59\") " pod="openshift-image-registry/node-ca-dffdw" Jul 29 00:35:50.806443 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806331 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-run-systemd\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.806443 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806372 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-multus-cni-dir\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.806443 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806374 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz88p\" (UniqueName: \"kubernetes.io/projected/e60adc84-fa8b-4053-9334-fc85a835ccf7-kube-api-access-jz88p\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.806443 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806422 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnmkj\" (UniqueName: \"kubernetes.io/projected/41c070b5-13d1-47ea-a5e4-50f519e7e647-kube-api-access-tnmkj\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806451 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-sysctl-conf\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806476 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da105473-e895-4852-a94f-a9525dc75801-tmp\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806501 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806531 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806559 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/703966f4-444e-4b3b-854d-3a7b67259320-konnectivity-ca\") pod \"konnectivity-agent-cjfg6\" (UID: \"703966f4-444e-4b3b-854d-3a7b67259320\") " pod="kube-system/konnectivity-agent-cjfg6" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806586 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb151e18-2bf5-4098-a956-829f7bbf9e4e-cni-binary-copy\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806610 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-lib-modules\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806632 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-sys-fs\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806659 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2jg7\" (UniqueName: \"kubernetes.io/projected/52f77a28-1b93-4bcc-8047-47f7073c48c2-kube-api-access-c2jg7\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806687 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7l5h\" (UniqueName: \"kubernetes.io/projected/b5888d04-a838-4a13-ac06-5ef697b5ee59-kube-api-access-c7l5h\") pod \"node-ca-dffdw\" (UID: \"b5888d04-a838-4a13-ac06-5ef697b5ee59\") " pod="openshift-image-registry/node-ca-dffdw" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806699 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-run-systemd\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806714 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e60adc84-fa8b-4053-9334-fc85a835ccf7-env-overrides\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806740 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-var-lib-cni-multus\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806763 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41c070b5-13d1-47ea-a5e4-50f519e7e647-multus-daemon-config\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806790 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-systemd\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806813 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-dbus\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:50.806979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806838 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-systemd-units\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806864 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb151e18-2bf5-4098-a956-829f7bbf9e4e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806893 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9z78\" (UniqueName: \"kubernetes.io/projected/da105473-e895-4852-a94f-a9525dc75801-kube-api-access-v9z78\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806919 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806937 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb151e18-2bf5-4098-a956-829f7bbf9e4e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806948 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-run-netns\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806954 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb151e18-2bf5-4098-a956-829f7bbf9e4e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807000 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-var-lib-cni-multus\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.806937 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41c070b5-13d1-47ea-a5e4-50f519e7e647-cni-binary-copy\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807077 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-dbus\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807088 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e60adc84-fa8b-4053-9334-fc85a835ccf7-ovn-node-metrics-cert\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807137 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-os-release\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807179 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-run\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807208 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-registration-dir\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807235 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-device-dir\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807277 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/703966f4-444e-4b3b-854d-3a7b67259320-konnectivity-ca\") pod \"konnectivity-agent-cjfg6\" (UID: \"703966f4-444e-4b3b-854d-3a7b67259320\") " pod="kube-system/konnectivity-agent-cjfg6" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807282 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8529g\" (UniqueName: \"kubernetes.io/projected/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-kube-api-access-8529g\") pod \"network-metrics-daemon-mgvr7\" (UID: \"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c\") " pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:35:50.807559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807309 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-sysctl-d\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807337 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-etc-openvswitch\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807353 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb151e18-2bf5-4098-a956-829f7bbf9e4e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807378 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807385 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-etc-openvswitch\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807465 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-os-release\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807572 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41c070b5-13d1-47ea-a5e4-50f519e7e647-multus-daemon-config\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807598 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-run-netns\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807624 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-systemd-units\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807629 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-sys\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807650 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e60adc84-fa8b-4053-9334-fc85a835ccf7-env-overrides\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807664 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e60adc84-fa8b-4053-9334-fc85a835ccf7-ovnkube-config\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807682 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-run-openvswitch\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807699 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-run-ovn\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807726 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj8ps\" (UniqueName: \"kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps\") pod \"network-check-target-2kc2k\" (UID: \"c56793a6-2612-4e39-84fd-5c357c49066a\") " pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807752 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-run-openvswitch\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807741 2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807754 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-kubernetes\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.808080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807825 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqqrn\" (UniqueName: \"kubernetes.io/projected/cb151e18-2bf5-4098-a956-829f7bbf9e4e-kube-api-access-cqqrn\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807857 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da105473-e895-4852-a94f-a9525dc75801-etc-tuned\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807880 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb151e18-2bf5-4098-a956-829f7bbf9e4e-cni-binary-copy\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807892 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-var-lib-openvswitch\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807934 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-run-ovn\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807940 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-log-socket\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.807971 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-var-lib-openvswitch\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808013 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-log-socket\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808033 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e60adc84-fa8b-4053-9334-fc85a835ccf7-ovnkube-script-lib\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808083 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-run-k8s-cni-cncf-io\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808125 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-multus-conf-dir\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808128 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e60adc84-fa8b-4053-9334-fc85a835ccf7-ovnkube-config\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808145 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-run-k8s-cni-cncf-io\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808153 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-kubelet-config\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808195 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a6daeacc-5a05-4ab1-8966-dd67c27e49d2-iptables-alerter-script\") pod \"iptables-alerter-kwth9\" (UID: \"a6daeacc-5a05-4ab1-8966-dd67c27e49d2\") " pod="openshift-network-operator/iptables-alerter-kwth9" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808203 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-kubelet-config\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808221 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6daeacc-5a05-4ab1-8966-dd67c27e49d2-host-slash\") pod \"iptables-alerter-kwth9\" (UID: \"a6daeacc-5a05-4ab1-8966-dd67c27e49d2\") " pod="openshift-network-operator/iptables-alerter-kwth9" Jul 29 00:35:50.809007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808162 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-multus-conf-dir\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808248 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-kubelet\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808275 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/20137ae0-172b-4165-9f8f-87bb7df1fd80-hosts-file\") pod \"node-resolver-vpqgd\" (UID: \"20137ae0-172b-4165-9f8f-87bb7df1fd80\") " pod="openshift-dns/node-resolver-vpqgd" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808299 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-hostroot\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808248 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6daeacc-5a05-4ab1-8966-dd67c27e49d2-host-slash\") pod \"iptables-alerter-kwth9\" (UID: \"a6daeacc-5a05-4ab1-8966-dd67c27e49d2\") " pod="openshift-network-operator/iptables-alerter-kwth9" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808336 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-run-multus-certs\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808362 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-slash\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808371 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-kubelet\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808386 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-cni-bin\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808412 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20137ae0-172b-4165-9f8f-87bb7df1fd80-tmp-dir\") pod \"node-resolver-vpqgd\" (UID: \"20137ae0-172b-4165-9f8f-87bb7df1fd80\") " pod="openshift-dns/node-resolver-vpqgd" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808418 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-run-multus-certs\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808421 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-hostroot\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808435 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-cnibin\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808462 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-multus-socket-dir-parent\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808471 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-slash\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808455 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-cni-bin\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808487 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-etc-kubernetes\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808516 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-modprobe-d\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.809813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808531 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-cnibin\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808540 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-sysconfig\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808568 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-run-ovn-kubernetes\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808579 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-multus-socket-dir-parent\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808386 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/20137ae0-172b-4165-9f8f-87bb7df1fd80-hosts-file\") pod \"node-resolver-vpqgd\" (UID: \"20137ae0-172b-4165-9f8f-87bb7df1fd80\") " pod="openshift-dns/node-resolver-vpqgd" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808595 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/703966f4-444e-4b3b-854d-3a7b67259320-agent-certs\") pod \"konnectivity-agent-cjfg6\" (UID: \"703966f4-444e-4b3b-854d-3a7b67259320\") " pod="kube-system/konnectivity-agent-cjfg6" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808621 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-var-lib-cni-bin\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808638 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-etc-kubernetes\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808648 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-socket-dir\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808681 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5888d04-a838-4a13-ac06-5ef697b5ee59-host\") pod \"node-ca-dffdw\" (UID: \"b5888d04-a838-4a13-ac06-5ef697b5ee59\") " pod="openshift-image-registry/node-ca-dffdw" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808707 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808703 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-host-run-ovn-kubernetes\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808732 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj8fk\" (UniqueName: \"kubernetes.io/projected/a6daeacc-5a05-4ab1-8966-dd67c27e49d2-kube-api-access-qj8fk\") pod \"iptables-alerter-kwth9\" (UID: \"a6daeacc-5a05-4ab1-8966-dd67c27e49d2\") " pod="openshift-network-operator/iptables-alerter-kwth9" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808769 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-var-lib-cni-bin\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808779 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a6daeacc-5a05-4ab1-8966-dd67c27e49d2-iptables-alerter-script\") pod \"iptables-alerter-kwth9\" (UID: \"a6daeacc-5a05-4ab1-8966-dd67c27e49d2\") " pod="openshift-network-operator/iptables-alerter-kwth9" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808782 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jttd\" (UniqueName: \"kubernetes.io/projected/20137ae0-172b-4165-9f8f-87bb7df1fd80-kube-api-access-2jttd\") pod \"node-resolver-vpqgd\" (UID: \"20137ae0-172b-4165-9f8f-87bb7df1fd80\") " pod="openshift-dns/node-resolver-vpqgd" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808819 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e60adc84-fa8b-4053-9334-fc85a835ccf7-ovnkube-script-lib\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.810597 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808827 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb151e18-2bf5-4098-a956-829f7bbf9e4e-system-cni-dir\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.811461 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808839 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20137ae0-172b-4165-9f8f-87bb7df1fd80-tmp-dir\") pod \"node-resolver-vpqgd\" (UID: \"20137ae0-172b-4165-9f8f-87bb7df1fd80\") " pod="openshift-dns/node-resolver-vpqgd" Jul 29 00:35:50.811461 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808859 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb151e18-2bf5-4098-a956-829f7bbf9e4e-cnibin\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.811461 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:50.808891 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 00:35:50.811461 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808894 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb151e18-2bf5-4098-a956-829f7bbf9e4e-system-cni-dir\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.811461 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808931 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-system-cni-dir\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.811461 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:50.808985 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret podName:a50c1131-277e-4dd2-9fdd-dfd5d5aabdee nodeName:}" failed. No retries permitted until 2026-07-29 00:35:51.308943773 +0000 UTC m=+2.968169668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret") pod "global-pull-secret-syncer-jghtb" (UID: "a50c1131-277e-4dd2-9fdd-dfd5d5aabdee") : object "kube-system"/"original-pull-secret" not registered Jul 29 00:35:50.811461 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.808998 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-system-cni-dir\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.811461 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.809009 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-run-netns\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.811461 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.809039 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-var-lib-kubelet\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.811461 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.809046 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb151e18-2bf5-4098-a956-829f7bbf9e4e-cnibin\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.811461 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.809076 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-node-log\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.811461 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.809094 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-var-lib-kubelet\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.811461 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.809131 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41c070b5-13d1-47ea-a5e4-50f519e7e647-host-run-netns\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.811461 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.809165 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e60adc84-fa8b-4053-9334-fc85a835ccf7-node-log\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.813045 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.812764 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e60adc84-fa8b-4053-9334-fc85a835ccf7-ovn-node-metrics-cert\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.813045 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.812893 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/703966f4-444e-4b3b-854d-3a7b67259320-agent-certs\") pod \"konnectivity-agent-cjfg6\" (UID: \"703966f4-444e-4b3b-854d-3a7b67259320\") " pod="kube-system/konnectivity-agent-cjfg6" Jul 29 00:35:50.813959 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:50.813938 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 00:35:50.814055 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:50.813968 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 00:35:50.814055 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:50.813982 2558 projected.go:196] Error preparing data for projected volume kube-api-access-wj8ps for pod openshift-network-diagnostics/network-check-target-2kc2k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 00:35:50.814055 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:50.814033 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps podName:c56793a6-2612-4e39-84fd-5c357c49066a nodeName:}" failed. No retries permitted until 2026-07-29 00:35:51.314017399 +0000 UTC m=+2.973243290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wj8ps" (UniqueName: "kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps") pod "network-check-target-2kc2k" (UID: "c56793a6-2612-4e39-84fd-5c357c49066a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 00:35:50.816273 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.816252 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqqrn\" (UniqueName: \"kubernetes.io/projected/cb151e18-2bf5-4098-a956-829f7bbf9e4e-kube-api-access-cqqrn\") pod \"multus-additional-cni-plugins-c527k\" (UID: \"cb151e18-2bf5-4098-a956-829f7bbf9e4e\") " pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:50.816799 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.816775 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz88p\" (UniqueName: \"kubernetes.io/projected/e60adc84-fa8b-4053-9334-fc85a835ccf7-kube-api-access-jz88p\") pod \"ovnkube-node-mtt7s\" (UID: \"e60adc84-fa8b-4053-9334-fc85a835ccf7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:50.818650 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.818179 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnmkj\" (UniqueName: \"kubernetes.io/projected/41c070b5-13d1-47ea-a5e4-50f519e7e647-kube-api-access-tnmkj\") pod \"multus-w8sxh\" (UID: \"41c070b5-13d1-47ea-a5e4-50f519e7e647\") " pod="openshift-multus/multus-w8sxh" Jul 29 00:35:50.818650 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.818609 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jttd\" (UniqueName: \"kubernetes.io/projected/20137ae0-172b-4165-9f8f-87bb7df1fd80-kube-api-access-2jttd\") pod \"node-resolver-vpqgd\" (UID: \"20137ae0-172b-4165-9f8f-87bb7df1fd80\") " pod="openshift-dns/node-resolver-vpqgd" Jul 29 00:35:50.819772 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.819747 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj8fk\" (UniqueName: \"kubernetes.io/projected/a6daeacc-5a05-4ab1-8966-dd67c27e49d2-kube-api-access-qj8fk\") pod \"iptables-alerter-kwth9\" (UID: \"a6daeacc-5a05-4ab1-8966-dd67c27e49d2\") " pod="openshift-network-operator/iptables-alerter-kwth9" Jul 29 00:35:50.910091 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910054 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9z78\" (UniqueName: \"kubernetes.io/projected/da105473-e895-4852-a94f-a9525dc75801-kube-api-access-v9z78\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.910234 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910113 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.910234 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910153 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-run\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.910234 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910185 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-registration-dir\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.910234 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910214 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-device-dir\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.910496 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910265 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8529g\" (UniqueName: \"kubernetes.io/projected/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-kube-api-access-8529g\") pod \"network-metrics-daemon-mgvr7\" (UID: \"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c\") " pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:35:50.910496 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910313 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-sysctl-d\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.910496 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910377 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-sys\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.910638 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910515 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-kubernetes\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.910638 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910549 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da105473-e895-4852-a94f-a9525dc75801-etc-tuned\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.910638 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910599 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-modprobe-d\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.910774 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910659 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-sysconfig\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.910774 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910697 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-socket-dir\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.910774 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910741 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5888d04-a838-4a13-ac06-5ef697b5ee59-host\") pod \"node-ca-dffdw\" (UID: \"b5888d04-a838-4a13-ac06-5ef697b5ee59\") " pod="openshift-image-registry/node-ca-dffdw" Jul 29 00:35:50.910916 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910800 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-var-lib-kubelet\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.910916 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910830 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-host\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.911010 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910948 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs\") pod \"network-metrics-daemon-mgvr7\" (UID: \"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c\") " pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:35:50.911010 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.910985 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b5888d04-a838-4a13-ac06-5ef697b5ee59-serviceca\") pod \"node-ca-dffdw\" (UID: \"b5888d04-a838-4a13-ac06-5ef697b5ee59\") " pod="openshift-image-registry/node-ca-dffdw" Jul 29 00:35:50.911162 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.911017 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-sysctl-conf\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.911162 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.911061 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da105473-e895-4852-a94f-a9525dc75801-tmp\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.911162 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.911123 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.911301 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.911173 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-lib-modules\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.911301 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.911206 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-sys-fs\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.911301 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.911281 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2jg7\" (UniqueName: \"kubernetes.io/projected/52f77a28-1b93-4bcc-8047-47f7073c48c2-kube-api-access-c2jg7\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.911503 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.911316 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7l5h\" (UniqueName: \"kubernetes.io/projected/b5888d04-a838-4a13-ac06-5ef697b5ee59-kube-api-access-c7l5h\") pod \"node-ca-dffdw\" (UID: \"b5888d04-a838-4a13-ac06-5ef697b5ee59\") " pod="openshift-image-registry/node-ca-dffdw" Jul 29 00:35:50.911503 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.911354 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-systemd\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.911603 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.911524 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-systemd\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.911603 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.911583 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-socket-dir\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.911603 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.911605 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5888d04-a838-4a13-ac06-5ef697b5ee59-host\") pod \"node-ca-dffdw\" (UID: \"b5888d04-a838-4a13-ac06-5ef697b5ee59\") " pod="openshift-image-registry/node-ca-dffdw" Jul 29 00:35:50.911777 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.911660 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-var-lib-kubelet\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.911777 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.911746 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-host\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.911865 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:50.911844 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 00:35:50.912026 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:50.912009 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs podName:a00c6eb4-e6cb-4486-949f-f4f4a7ba538c nodeName:}" failed. No retries permitted until 2026-07-29 00:35:51.411988662 +0000 UTC m=+3.071214559 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs") pod "network-metrics-daemon-mgvr7" (UID: "a00c6eb4-e6cb-4486-949f-f4f4a7ba538c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 00:35:50.912106 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.912010 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.912106 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.912099 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-run\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.912226 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.912166 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-registration-dir\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.912280 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.912260 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-device-dir\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.912707 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.912670 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-sysctl-d\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.912790 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.912739 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b5888d04-a838-4a13-ac06-5ef697b5ee59-serviceca\") pod \"node-ca-dffdw\" (UID: \"b5888d04-a838-4a13-ac06-5ef697b5ee59\") " pod="openshift-image-registry/node-ca-dffdw" Jul 29 00:35:50.912790 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.912761 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-sys\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.912909 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.912807 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-kubernetes\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.912909 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.912885 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-sysctl-conf\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.915285 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.913314 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-sysconfig\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.915285 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.913474 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-etc-modprobe-d\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.915285 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.913574 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-sys-fs\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.915285 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.913636 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/52f77a28-1b93-4bcc-8047-47f7073c48c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.915285 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.913748 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da105473-e895-4852-a94f-a9525dc75801-lib-modules\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.917103 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.917073 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da105473-e895-4852-a94f-a9525dc75801-etc-tuned\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.919072 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.919031 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da105473-e895-4852-a94f-a9525dc75801-tmp\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.922304 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.922226 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8529g\" (UniqueName: \"kubernetes.io/projected/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-kube-api-access-8529g\") pod \"network-metrics-daemon-mgvr7\" (UID: \"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c\") " pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:35:50.923600 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.923545 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9z78\" (UniqueName: \"kubernetes.io/projected/da105473-e895-4852-a94f-a9525dc75801-kube-api-access-v9z78\") pod \"tuned-fjv69\" (UID: \"da105473-e895-4852-a94f-a9525dc75801\") " pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:50.925520 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.925477 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7l5h\" (UniqueName: \"kubernetes.io/projected/b5888d04-a838-4a13-ac06-5ef697b5ee59-kube-api-access-c7l5h\") pod \"node-ca-dffdw\" (UID: \"b5888d04-a838-4a13-ac06-5ef697b5ee59\") " pod="openshift-image-registry/node-ca-dffdw" Jul 29 00:35:50.926335 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.926312 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2jg7\" (UniqueName: \"kubernetes.io/projected/52f77a28-1b93-4bcc-8047-47f7073c48c2-kube-api-access-c2jg7\") pod \"aws-ebs-csi-driver-node-fdmmn\" (UID: \"52f77a28-1b93-4bcc-8047-47f7073c48c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:50.996911 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:50.996820 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w8sxh" Jul 29 00:35:51.007047 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.007022 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cjfg6" Jul 29 00:35:51.017829 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.017805 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vpqgd" Jul 29 00:35:51.025396 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.025212 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c527k" Jul 29 00:35:51.034115 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.034072 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kwth9" Jul 29 00:35:51.042027 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.042007 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:35:51.051013 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.050989 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fjv69" Jul 29 00:35:51.058067 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.058043 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" Jul 29 00:35:51.063108 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.063087 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dffdw" Jul 29 00:35:51.084690 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:51.084666 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5888d04_a838_4a13_ac06_5ef697b5ee59.slice/crio-7668ac995c9df8d97d776fa91be171321ed51a9a2aeb08ab54ec2b5e140f9132 WatchSource:0}: Error finding container 7668ac995c9df8d97d776fa91be171321ed51a9a2aeb08ab54ec2b5e140f9132: Status 404 returned error can't find the container with id 7668ac995c9df8d97d776fa91be171321ed51a9a2aeb08ab54ec2b5e140f9132 Jul 29 00:35:51.114020 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:51.113984 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda105473_e895_4852_a94f_a9525dc75801.slice/crio-b2d877e0c6623efadae785438bc2f56ccb4359299a3890de0a3830ee491f3635 WatchSource:0}: Error finding container b2d877e0c6623efadae785438bc2f56ccb4359299a3890de0a3830ee491f3635: Status 404 returned error can't find the container with id b2d877e0c6623efadae785438bc2f56ccb4359299a3890de0a3830ee491f3635 Jul 29 00:35:51.114910 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:51.114856 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode60adc84_fa8b_4053_9334_fc85a835ccf7.slice/crio-9ce6ea0cb369fcaed537a9ebd3bf3f875af2738a21bfcc820bf00629fcf84722 WatchSource:0}: Error finding container 9ce6ea0cb369fcaed537a9ebd3bf3f875af2738a21bfcc820bf00629fcf84722: Status 404 returned error can't find the container with id 9ce6ea0cb369fcaed537a9ebd3bf3f875af2738a21bfcc820bf00629fcf84722 Jul 29 00:35:51.116526 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:51.116443 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f77a28_1b93_4bcc_8047_47f7073c48c2.slice/crio-2ae24fc3a6756c9a88837be72f08a029ade060cc206f328e00d353ecd3bc88e8 WatchSource:0}: Error finding container 2ae24fc3a6756c9a88837be72f08a029ade060cc206f328e00d353ecd3bc88e8: Status 404 returned error can't find the container with id 2ae24fc3a6756c9a88837be72f08a029ade060cc206f328e00d353ecd3bc88e8 Jul 29 00:35:51.116955 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:35:51.116936 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c070b5_13d1_47ea_a5e4_50f519e7e647.slice/crio-d30273fcbd88e6fb6eff9bcf81e6799c617c9163fbc667fbf59ed89545755eed WatchSource:0}: Error finding container d30273fcbd88e6fb6eff9bcf81e6799c617c9163fbc667fbf59ed89545755eed: Status 404 returned error can't find the container with id d30273fcbd88e6fb6eff9bcf81e6799c617c9163fbc667fbf59ed89545755eed Jul 29 00:35:51.148892 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.148867 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 00:35:51.313646 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.313616 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:51.313810 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:51.313789 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 00:35:51.313880 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:51.313863 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret podName:a50c1131-277e-4dd2-9fdd-dfd5d5aabdee nodeName:}" failed. No retries permitted until 2026-07-29 00:35:52.313843805 +0000 UTC m=+3.973069679 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret") pod "global-pull-secret-syncer-jghtb" (UID: "a50c1131-277e-4dd2-9fdd-dfd5d5aabdee") : object "kube-system"/"original-pull-secret" not registered Jul 29 00:35:51.414995 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.414966 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs\") pod \"network-metrics-daemon-mgvr7\" (UID: \"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c\") " pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:35:51.415132 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.415020 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj8ps\" (UniqueName: \"kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps\") pod \"network-check-target-2kc2k\" (UID: \"c56793a6-2612-4e39-84fd-5c357c49066a\") " pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:35:51.415196 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:51.415153 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 00:35:51.415196 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:51.415173 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 00:35:51.415196 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:51.415187 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 00:35:51.415325 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:51.415198 2558 projected.go:196] Error preparing data for projected volume kube-api-access-wj8ps for pod openshift-network-diagnostics/network-check-target-2kc2k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 00:35:51.415325 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:51.415239 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs podName:a00c6eb4-e6cb-4486-949f-f4f4a7ba538c nodeName:}" failed. No retries permitted until 2026-07-29 00:35:52.415213873 +0000 UTC m=+4.074439764 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs") pod "network-metrics-daemon-mgvr7" (UID: "a00c6eb4-e6cb-4486-949f-f4f4a7ba538c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 00:35:51.415325 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:51.415260 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps podName:c56793a6-2612-4e39-84fd-5c357c49066a nodeName:}" failed. No retries permitted until 2026-07-29 00:35:52.415249809 +0000 UTC m=+4.074475688 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wj8ps" (UniqueName: "kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps") pod "network-check-target-2kc2k" (UID: "c56793a6-2612-4e39-84fd-5c357c49066a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 00:35:51.750602 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.750510 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 00:30:51 +0000 UTC" deadline="2028-04-30 23:48:14.707422626 +0000 UTC" Jul 29 00:35:51.750602 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.750554 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15407h12m22.956873837s" Jul 29 00:35:51.791028 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.790988 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cjfg6" event={"ID":"703966f4-444e-4b3b-854d-3a7b67259320","Type":"ContainerStarted","Data":"6c41d578d8e692b2b30a9490013ff869ddf35327309d2b2aae7c58a582497c87"} Jul 29 00:35:51.794763 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.793809 2558 generic.go:356] "Generic (PLEG): container finished" podID="1f0d2324c5d97061fc92a237a96dbdcf" containerID="485bb7cbfedf0c83132618790477797c8e7874db467ed2d5711277e0e2d6cf94" exitCode=0 Jul 29 00:35:51.794763 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.794528 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" event={"ID":"1f0d2324c5d97061fc92a237a96dbdcf","Type":"ContainerDied","Data":"485bb7cbfedf0c83132618790477797c8e7874db467ed2d5711277e0e2d6cf94"} Jul 29 00:35:51.796687 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.796656 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" event={"ID":"e60adc84-fa8b-4053-9334-fc85a835ccf7","Type":"ContainerStarted","Data":"9ce6ea0cb369fcaed537a9ebd3bf3f875af2738a21bfcc820bf00629fcf84722"} Jul 29 00:35:51.798977 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.798907 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w8sxh" event={"ID":"41c070b5-13d1-47ea-a5e4-50f519e7e647","Type":"ContainerStarted","Data":"d30273fcbd88e6fb6eff9bcf81e6799c617c9163fbc667fbf59ed89545755eed"} Jul 29 00:35:51.800380 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.800355 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c527k" event={"ID":"cb151e18-2bf5-4098-a956-829f7bbf9e4e","Type":"ContainerStarted","Data":"25f333a6d28fb49b66201a80196b7b9c3088b32c0e5b21a84a7ec0ebb8b7d827"} Jul 29 00:35:51.802080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.802017 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dffdw" event={"ID":"b5888d04-a838-4a13-ac06-5ef697b5ee59","Type":"ContainerStarted","Data":"7668ac995c9df8d97d776fa91be171321ed51a9a2aeb08ab54ec2b5e140f9132"} Jul 29 00:35:51.803803 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.803777 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" event={"ID":"52f77a28-1b93-4bcc-8047-47f7073c48c2","Type":"ContainerStarted","Data":"2ae24fc3a6756c9a88837be72f08a029ade060cc206f328e00d353ecd3bc88e8"} Jul 29 00:35:51.805460 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.805156 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fjv69" event={"ID":"da105473-e895-4852-a94f-a9525dc75801","Type":"ContainerStarted","Data":"b2d877e0c6623efadae785438bc2f56ccb4359299a3890de0a3830ee491f3635"} Jul 29 00:35:51.807055 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.807023 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kwth9" event={"ID":"a6daeacc-5a05-4ab1-8966-dd67c27e49d2","Type":"ContainerStarted","Data":"4e2dd4281c8272017d23c99a7e902c78cc98172994e0361a288779e72d7d4130"} Jul 29 00:35:51.810320 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:51.810294 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vpqgd" event={"ID":"20137ae0-172b-4165-9f8f-87bb7df1fd80","Type":"ContainerStarted","Data":"205e69fa1b7ca9b4726d16aa9e078a0b095e65f65f237accb2918382592cc48d"} Jul 29 00:35:52.328173 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:52.327576 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:52.328173 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:52.327754 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 00:35:52.328173 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:52.327817 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret podName:a50c1131-277e-4dd2-9fdd-dfd5d5aabdee nodeName:}" failed. No retries permitted until 2026-07-29 00:35:54.327799058 +0000 UTC m=+5.987024937 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret") pod "global-pull-secret-syncer-jghtb" (UID: "a50c1131-277e-4dd2-9fdd-dfd5d5aabdee") : object "kube-system"/"original-pull-secret" not registered Jul 29 00:35:52.428580 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:52.428537 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs\") pod \"network-metrics-daemon-mgvr7\" (UID: \"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c\") " pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:35:52.428715 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:52.428608 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj8ps\" (UniqueName: \"kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps\") pod \"network-check-target-2kc2k\" (UID: \"c56793a6-2612-4e39-84fd-5c357c49066a\") " pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:35:52.428782 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:52.428765 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 00:35:52.428836 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:52.428790 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 00:35:52.428836 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:52.428803 2558 projected.go:196] Error preparing data for projected volume kube-api-access-wj8ps for pod openshift-network-diagnostics/network-check-target-2kc2k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 00:35:52.428953 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:52.428859 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps podName:c56793a6-2612-4e39-84fd-5c357c49066a nodeName:}" failed. No retries permitted until 2026-07-29 00:35:54.428839192 +0000 UTC m=+6.088065070 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wj8ps" (UniqueName: "kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps") pod "network-check-target-2kc2k" (UID: "c56793a6-2612-4e39-84fd-5c357c49066a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 00:35:52.429022 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:52.428960 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 00:35:52.429022 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:52.429012 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs podName:a00c6eb4-e6cb-4486-949f-f4f4a7ba538c nodeName:}" failed. No retries permitted until 2026-07-29 00:35:54.428992335 +0000 UTC m=+6.088218212 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs") pod "network-metrics-daemon-mgvr7" (UID: "a00c6eb4-e6cb-4486-949f-f4f4a7ba538c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 00:35:52.751777 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:52.751464 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 00:30:51 +0000 UTC" deadline="2027-12-31 18:46:25.040789094 +0000 UTC" Jul 29 00:35:52.751777 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:52.751750 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12498h10m32.289047403s" Jul 29 00:35:52.784530 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:52.784500 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:52.784692 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:52.784637 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:35:52.785485 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:52.785065 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:35:52.785485 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:52.785180 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:35:52.785485 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:52.785360 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:35:52.785485 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:52.785450 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:35:52.821835 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:52.821802 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal" event={"ID":"fe0818b1e6c45ae97ec4ef99e813669f","Type":"ContainerStarted","Data":"d42f7aed4e40aedaa156054faf903a24ad88e367ec3715fafb833832b6eb7cf8"} Jul 29 00:35:52.829333 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:52.829201 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" event={"ID":"1f0d2324c5d97061fc92a237a96dbdcf","Type":"ContainerStarted","Data":"f04fa06a999f16067bfd2c441ac8f3cd58d0c3d7aad037eb998f65557cb7eb2e"} Jul 29 00:35:52.853576 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:52.853519 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-11.ec2.internal" podStartSLOduration=2.853500361 podStartE2EDuration="2.853500361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:35:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:35:52.838268896 +0000 UTC m=+4.497494792" watchObservedRunningTime="2026-07-29 00:35:52.853500361 +0000 UTC m=+4.512726259" Jul 29 00:35:52.854454 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:52.854417 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-11.ec2.internal" podStartSLOduration=2.854401261 podStartE2EDuration="2.854401261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:35:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:35:52.853288216 +0000 UTC m=+4.512514112" watchObservedRunningTime="2026-07-29 00:35:52.854401261 +0000 UTC m=+4.513627148" Jul 29 00:35:54.346962 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:54.346263 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:54.346962 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:54.346484 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 00:35:54.346962 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:54.346557 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret podName:a50c1131-277e-4dd2-9fdd-dfd5d5aabdee nodeName:}" failed. No retries permitted until 2026-07-29 00:35:58.346535705 +0000 UTC m=+10.005761579 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret") pod "global-pull-secret-syncer-jghtb" (UID: "a50c1131-277e-4dd2-9fdd-dfd5d5aabdee") : object "kube-system"/"original-pull-secret" not registered Jul 29 00:35:54.448493 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:54.447581 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj8ps\" (UniqueName: \"kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps\") pod \"network-check-target-2kc2k\" (UID: \"c56793a6-2612-4e39-84fd-5c357c49066a\") " pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:35:54.448493 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:54.447676 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs\") pod \"network-metrics-daemon-mgvr7\" (UID: \"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c\") " pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:35:54.448493 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:54.447816 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 00:35:54.448493 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:54.447881 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs podName:a00c6eb4-e6cb-4486-949f-f4f4a7ba538c nodeName:}" failed. No retries permitted until 2026-07-29 00:35:58.44786188 +0000 UTC m=+10.107087778 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs") pod "network-metrics-daemon-mgvr7" (UID: "a00c6eb4-e6cb-4486-949f-f4f4a7ba538c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 00:35:54.448493 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:54.448355 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 00:35:54.448493 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:54.448376 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 00:35:54.448493 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:54.448406 2558 projected.go:196] Error preparing data for projected volume kube-api-access-wj8ps for pod openshift-network-diagnostics/network-check-target-2kc2k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 00:35:54.448493 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:54.448457 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps podName:c56793a6-2612-4e39-84fd-5c357c49066a nodeName:}" failed. No retries permitted until 2026-07-29 00:35:58.448441043 +0000 UTC m=+10.107666922 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wj8ps" (UniqueName: "kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps") pod "network-check-target-2kc2k" (UID: "c56793a6-2612-4e39-84fd-5c357c49066a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 00:35:54.786396 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:54.786234 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:35:54.786566 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:54.786408 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:54.786566 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:54.786441 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:35:54.786566 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:54.786499 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:35:54.786566 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:54.786537 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:35:54.786787 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:54.786599 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:35:56.783475 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:56.783411 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:35:56.783982 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:56.783561 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:35:56.783982 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:56.783689 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:35:56.783982 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:56.783553 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:35:56.783982 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:56.783418 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:56.783982 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:56.783799 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:35:58.380306 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:58.380266 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:58.380821 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:58.380514 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 00:35:58.380821 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:58.380594 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret podName:a50c1131-277e-4dd2-9fdd-dfd5d5aabdee nodeName:}" failed. No retries permitted until 2026-07-29 00:36:06.380570246 +0000 UTC m=+18.039796144 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret") pod "global-pull-secret-syncer-jghtb" (UID: "a50c1131-277e-4dd2-9fdd-dfd5d5aabdee") : object "kube-system"/"original-pull-secret" not registered Jul 29 00:35:58.480994 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:58.480955 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj8ps\" (UniqueName: \"kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps\") pod \"network-check-target-2kc2k\" (UID: \"c56793a6-2612-4e39-84fd-5c357c49066a\") " pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:35:58.481177 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:58.481048 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs\") pod \"network-metrics-daemon-mgvr7\" (UID: \"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c\") " pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:35:58.481177 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:58.481145 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 00:35:58.481177 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:58.481172 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 00:35:58.481376 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:58.481186 2558 projected.go:196] Error preparing data for projected volume kube-api-access-wj8ps for pod openshift-network-diagnostics/network-check-target-2kc2k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 00:35:58.481376 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:58.481245 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps podName:c56793a6-2612-4e39-84fd-5c357c49066a nodeName:}" failed. No retries permitted until 2026-07-29 00:36:06.481225472 +0000 UTC m=+18.140451360 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wj8ps" (UniqueName: "kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps") pod "network-check-target-2kc2k" (UID: "c56793a6-2612-4e39-84fd-5c357c49066a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 00:35:58.481376 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:58.481244 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 00:35:58.481376 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:58.481289 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs podName:a00c6eb4-e6cb-4486-949f-f4f4a7ba538c nodeName:}" failed. No retries permitted until 2026-07-29 00:36:06.481277041 +0000 UTC m=+18.140502918 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs") pod "network-metrics-daemon-mgvr7" (UID: "a00c6eb4-e6cb-4486-949f-f4f4a7ba538c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 00:35:58.785542 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:58.784697 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:35:58.785542 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:58.784816 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:35:58.785542 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:58.785231 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:35:58.785542 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:58.785415 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:35:58.785895 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:35:58.785572 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:35:58.785895 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:35:58.785656 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:36:00.784839 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:00.784793 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:00.785376 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:00.784793 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:00.785376 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:00.784925 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:36:00.785376 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:00.785030 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:36:00.785376 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:00.784799 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:00.785376 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:00.785114 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:36:02.784037 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:02.783949 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:02.784504 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:02.783960 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:02.784504 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:02.784073 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:36:02.784504 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:02.783966 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:02.784504 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:02.784187 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:36:02.784504 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:02.784240 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:36:04.782916 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:04.782886 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:04.783420 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:04.782886 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:04.783420 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:04.783017 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:36:04.783420 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:04.782892 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:04.783420 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:04.783108 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:36:04.783420 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:04.783195 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:36:06.444053 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:06.444013 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:06.444593 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:06.444183 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 00:36:06.444593 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:06.444260 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret podName:a50c1131-277e-4dd2-9fdd-dfd5d5aabdee nodeName:}" failed. No retries permitted until 2026-07-29 00:36:22.444238193 +0000 UTC m=+34.103464064 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret") pod "global-pull-secret-syncer-jghtb" (UID: "a50c1131-277e-4dd2-9fdd-dfd5d5aabdee") : object "kube-system"/"original-pull-secret" not registered Jul 29 00:36:06.544742 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:06.544706 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj8ps\" (UniqueName: \"kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps\") pod \"network-check-target-2kc2k\" (UID: \"c56793a6-2612-4e39-84fd-5c357c49066a\") " pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:06.544946 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:06.544787 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs\") pod \"network-metrics-daemon-mgvr7\" (UID: \"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c\") " pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:06.544946 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:06.544897 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 00:36:06.544946 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:06.544902 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 00:36:06.544946 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:06.544930 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 00:36:06.544946 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:06.544943 2558 projected.go:196] Error preparing data for projected volume kube-api-access-wj8ps for pod openshift-network-diagnostics/network-check-target-2kc2k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 00:36:06.545186 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:06.544955 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs podName:a00c6eb4-e6cb-4486-949f-f4f4a7ba538c nodeName:}" failed. No retries permitted until 2026-07-29 00:36:22.544937123 +0000 UTC m=+34.204162995 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs") pod "network-metrics-daemon-mgvr7" (UID: "a00c6eb4-e6cb-4486-949f-f4f4a7ba538c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 00:36:06.545186 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:06.544993 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps podName:c56793a6-2612-4e39-84fd-5c357c49066a nodeName:}" failed. No retries permitted until 2026-07-29 00:36:22.544976544 +0000 UTC m=+34.204202418 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wj8ps" (UniqueName: "kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps") pod "network-check-target-2kc2k" (UID: "c56793a6-2612-4e39-84fd-5c357c49066a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 00:36:06.783668 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:06.783571 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:06.783668 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:06.783621 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:06.783668 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:06.783571 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:06.783903 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:06.783701 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:36:06.783903 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:06.783823 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:36:06.784008 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:06.783919 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:36:08.783713 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:08.783685 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:08.784140 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:08.783800 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:36:08.784140 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:08.783885 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:08.784140 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:08.783991 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:08.784140 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:08.784072 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:36:08.784140 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:08.784114 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:36:10.783402 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.783196 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:10.783685 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.783210 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:10.783685 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:10.783473 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:36:10.783685 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.783237 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:10.783685 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:10.783562 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:36:10.783685 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:10.783607 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:36:10.867794 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.867758 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" event={"ID":"52f77a28-1b93-4bcc-8047-47f7073c48c2","Type":"ContainerStarted","Data":"6a07e5ed0a567cb96129f45c1bb4af466bb3dc134a85074bfc129989fdf7e764"} Jul 29 00:36:10.869259 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.869229 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fjv69" event={"ID":"da105473-e895-4852-a94f-a9525dc75801","Type":"ContainerStarted","Data":"82d584a02d6875dba7f4196cf57098f32717ab24820a4929dc87715475aea24a"} Jul 29 00:36:10.870858 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.870829 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vpqgd" event={"ID":"20137ae0-172b-4165-9f8f-87bb7df1fd80","Type":"ContainerStarted","Data":"2c4e232bc5450e2e62f1e86d11a82fa42bdbf2935ff738d5aaac4f582c3fca15"} Jul 29 00:36:10.872648 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.872591 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cjfg6" event={"ID":"703966f4-444e-4b3b-854d-3a7b67259320","Type":"ContainerStarted","Data":"e6832744e35817e6e0c39e71c2190000202678cf359098c3403a01b0cf6efb4b"} Jul 29 00:36:10.874338 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.874294 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w8sxh" event={"ID":"41c070b5-13d1-47ea-a5e4-50f519e7e647","Type":"ContainerStarted","Data":"d554436f96885751adb4d4890e2c0b18090b4442e8970dfd04797b51a8204ec9"} Jul 29 00:36:10.875903 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.875884 2558 generic.go:356] "Generic (PLEG): container finished" podID="cb151e18-2bf5-4098-a956-829f7bbf9e4e" containerID="18409c90bc99d9f3534fcea6d57107e1c4f98e2f3496bf14bca280ebd1474ba2" exitCode=0 Jul 29 00:36:10.876017 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.875944 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c527k" event={"ID":"cb151e18-2bf5-4098-a956-829f7bbf9e4e","Type":"ContainerDied","Data":"18409c90bc99d9f3534fcea6d57107e1c4f98e2f3496bf14bca280ebd1474ba2"} Jul 29 00:36:10.877647 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.877627 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dffdw" event={"ID":"b5888d04-a838-4a13-ac06-5ef697b5ee59","Type":"ContainerStarted","Data":"30ae861cca160ffbd3bce35a06c82f26a37188d935146ac53d8264b1bb98ff48"} Jul 29 00:36:10.888279 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.888242 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fjv69" podStartSLOduration=2.457386142 podStartE2EDuration="21.888231846s" totalImagesPullingTime="19.430845704s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:35:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:10.888129068 +0000 UTC m=+22.547354962" watchObservedRunningTime="2026-07-29 00:36:10.888231846 +0000 UTC m=+22.547457739" Jul 29 00:36:10.902258 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.902219 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dffdw" podStartSLOduration=1.453022452 podStartE2EDuration="20.90220708s" totalImagesPullingTime="19.449184628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:35:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:10.90207896 +0000 UTC m=+22.561304853" watchObservedRunningTime="2026-07-29 00:36:10.90220708 +0000 UTC m=+22.561432972" Jul 29 00:36:10.923878 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.923842 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cjfg6" podStartSLOduration=10.584355731 podStartE2EDuration="21.923828221s" totalImagesPullingTime="11.33947249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:35:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:10.923506005 +0000 UTC m=+22.582731897" watchObservedRunningTime="2026-07-29 00:36:10.923828221 +0000 UTC m=+22.583054114" Jul 29 00:36:10.953607 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.953561 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-w8sxh" podStartSLOduration=2.5112970900000002 podStartE2EDuration="21.953546475s" totalImagesPullingTime="19.442249385s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:35:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:10.953220481 +0000 UTC m=+22.612446374" watchObservedRunningTime="2026-07-29 00:36:10.953546475 +0000 UTC m=+22.612772368" Jul 29 00:36:10.989778 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:10.989738 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vpqgd" podStartSLOduration=2.552727842 podStartE2EDuration="21.98972341s" totalImagesPullingTime="19.436995568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:35:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:10.968813938 +0000 UTC m=+22.628039828" watchObservedRunningTime="2026-07-29 00:36:10.98972341 +0000 UTC m=+22.648949303" Jul 29 00:36:11.883143 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:11.882955 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 00:36:11.884117 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:11.883565 2558 generic.go:356] "Generic (PLEG): container finished" podID="e60adc84-fa8b-4053-9334-fc85a835ccf7" containerID="f9240b3d6c0e97b4d1a75c14eec5aa38b1a00245de60fbb0f5ef2f3030d8bccd" exitCode=1 Jul 29 00:36:11.884117 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:11.883700 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" event={"ID":"e60adc84-fa8b-4053-9334-fc85a835ccf7","Type":"ContainerStarted","Data":"1debb0ccd97b66121fa5fa98691846f591a6a5af360820a254523f6242ddea2b"} Jul 29 00:36:11.884117 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:11.883813 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" event={"ID":"e60adc84-fa8b-4053-9334-fc85a835ccf7","Type":"ContainerStarted","Data":"933276886dcaa07477f5fd59028e6709fd6fb7f55f2d934e9af5637b3ffec1bf"} Jul 29 00:36:11.884117 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:11.883847 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" event={"ID":"e60adc84-fa8b-4053-9334-fc85a835ccf7","Type":"ContainerStarted","Data":"316cb1d2347e1fda13891c543393c4280d7e1176597434e4324f1558bdc6ffcb"} Jul 29 00:36:11.884117 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:11.883875 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" event={"ID":"e60adc84-fa8b-4053-9334-fc85a835ccf7","Type":"ContainerStarted","Data":"3d7720c0bdbc9a911254a74449291aec1083278e2a1d65d72f02faa12b94a0a1"} Jul 29 00:36:11.884117 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:11.883909 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" event={"ID":"e60adc84-fa8b-4053-9334-fc85a835ccf7","Type":"ContainerDied","Data":"f9240b3d6c0e97b4d1a75c14eec5aa38b1a00245de60fbb0f5ef2f3030d8bccd"} Jul 29 00:36:11.884117 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:11.883955 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" event={"ID":"e60adc84-fa8b-4053-9334-fc85a835ccf7","Type":"ContainerStarted","Data":"e042979d75e56a494587538f2b98b07ab9572028dd6aeaef5b6eb0a5e40655a9"} Jul 29 00:36:12.148077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:12.148049 2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 29 00:36:12.758431 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:12.758211 2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T00:36:12.148069441Z","UUID":"db638885-52d3-4889-b92f-481ee339a7fb","Handler":null,"Name":"","Endpoint":""} Jul 29 00:36:12.760548 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:12.760519 2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 29 00:36:12.760717 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:12.760563 2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 29 00:36:12.783009 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:12.782977 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:12.783009 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:12.782995 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:12.783267 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:12.782978 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:12.783267 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:12.783103 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:36:12.783267 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:12.783194 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:36:12.783451 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:12.783280 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:36:12.888154 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:12.888121 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" event={"ID":"52f77a28-1b93-4bcc-8047-47f7073c48c2","Type":"ContainerStarted","Data":"2dedf844e44e2a5c3c8435812350756a69a146d634a63db8e98dfa44caa42368"} Jul 29 00:36:12.889874 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:12.889846 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kwth9" event={"ID":"a6daeacc-5a05-4ab1-8966-dd67c27e49d2","Type":"ContainerStarted","Data":"3747b3355dd4ff18cf2c96a17ee01b55d5eb255bb47011b6e2313b53d5f2f71f"} Jul 29 00:36:13.201635 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:13.201604 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cjfg6" Jul 29 00:36:13.202196 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:13.202176 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cjfg6" Jul 29 00:36:13.215336 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:13.215275 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kwth9" podStartSLOduration=4.777547993 podStartE2EDuration="24.215257492s" totalImagesPullingTime="19.437709499s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:35:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:12.904624808 +0000 UTC m=+24.563850702" watchObservedRunningTime="2026-07-29 00:36:13.215257492 +0000 UTC m=+24.874483386" Jul 29 00:36:13.894093 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:13.894057 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" event={"ID":"52f77a28-1b93-4bcc-8047-47f7073c48c2","Type":"ContainerStarted","Data":"416bcbb1b5226c2540b8b9d91d6e58542b420b637f07d69697ab6d3f25195547"} Jul 29 00:36:13.894574 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:13.894236 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cjfg6" Jul 29 00:36:13.895048 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:13.895023 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cjfg6" Jul 29 00:36:13.914196 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:13.914115 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdmmn" podStartSLOduration=3.228375658 podStartE2EDuration="24.914102485s" totalImagesPullingTime="21.685726827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:35:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:13.914010707 +0000 UTC m=+25.573236601" watchObservedRunningTime="2026-07-29 00:36:13.914102485 +0000 UTC m=+25.573328379" Jul 29 00:36:14.783155 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:14.782923 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:14.783339 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:14.782924 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:14.783339 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:14.783256 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:36:14.783451 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:14.783347 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:36:14.783451 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:14.782930 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:14.783534 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:14.783453 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:36:15.901052 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:15.901027 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 00:36:15.901642 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:15.901424 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" event={"ID":"e60adc84-fa8b-4053-9334-fc85a835ccf7","Type":"ContainerStarted","Data":"409a87b333f7f365ddf711f721403d97a24b5653be5d9aa6990e13cd984010c1"} Jul 29 00:36:15.903030 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:15.903006 2558 generic.go:356] "Generic (PLEG): container finished" podID="cb151e18-2bf5-4098-a956-829f7bbf9e4e" containerID="2db4528c02f0bed0fd35c98545703144011422680d035fa7679dd2457b96dd34" exitCode=0 Jul 29 00:36:15.903119 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:15.903095 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c527k" event={"ID":"cb151e18-2bf5-4098-a956-829f7bbf9e4e","Type":"ContainerDied","Data":"2db4528c02f0bed0fd35c98545703144011422680d035fa7679dd2457b96dd34"} Jul 29 00:36:16.783126 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:16.783101 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:16.783126 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:16.783101 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:16.783384 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:16.783255 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:36:16.783384 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:16.783299 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:16.783492 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:16.783381 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:36:16.783492 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:16.783438 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:36:17.911546 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:17.911368 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 00:36:17.912231 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:17.911927 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" event={"ID":"e60adc84-fa8b-4053-9334-fc85a835ccf7","Type":"ContainerStarted","Data":"f6beb0b483151bbe29ec60a73ac8813054667964563cdfc5f4f1fe263f53fecf"} Jul 29 00:36:17.912369 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:17.912304 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:36:17.912484 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:17.912398 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:36:17.912484 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:17.912440 2558 scope.go:117] "RemoveContainer" containerID="f9240b3d6c0e97b4d1a75c14eec5aa38b1a00245de60fbb0f5ef2f3030d8bccd" Jul 29 00:36:17.914227 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:17.914207 2558 generic.go:356] "Generic (PLEG): container finished" podID="cb151e18-2bf5-4098-a956-829f7bbf9e4e" containerID="4226ee5a7ff4d8adad2bc8363141d42de71f3c0ad5359c0505c82fa836a1863f" exitCode=0 Jul 29 00:36:17.914367 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:17.914249 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c527k" event={"ID":"cb151e18-2bf5-4098-a956-829f7bbf9e4e","Type":"ContainerDied","Data":"4226ee5a7ff4d8adad2bc8363141d42de71f3c0ad5359c0505c82fa836a1863f"} Jul 29 00:36:17.930470 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:17.930451 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:36:17.930615 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:17.930601 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:36:18.783722 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:18.783688 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:18.783933 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:18.783800 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:18.783933 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:18.783810 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:36:18.783933 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:18.783900 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:36:18.784102 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:18.783941 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:18.784102 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:18.784020 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:36:18.921912 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:18.921879 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 00:36:18.922337 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:18.922283 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" event={"ID":"e60adc84-fa8b-4053-9334-fc85a835ccf7","Type":"ContainerStarted","Data":"13d7362682deb7326a4b64758103783f08cdee7e484722fc4985a666488dd627"} Jul 29 00:36:18.922427 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:18.922414 2558 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 29 00:36:18.953705 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:18.953659 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" podStartSLOduration=10.14794834 podStartE2EDuration="29.953643088s" totalImagesPullingTime="19.805694748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:35:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:18.953134811 +0000 UTC m=+30.612360704" watchObservedRunningTime="2026-07-29 00:36:18.953643088 +0000 UTC m=+30.612868980" Jul 29 00:36:19.434670 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:19.434581 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jghtb"] Jul 29 00:36:19.434816 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:19.434744 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:19.434873 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:19.434851 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:36:19.437884 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:19.437854 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mgvr7"] Jul 29 00:36:19.438006 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:19.437975 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:19.438125 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:19.438079 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:36:19.442006 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:19.441986 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2kc2k"] Jul 29 00:36:19.442128 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:19.442078 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:19.442186 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:19.442158 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:36:19.927139 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:19.927111 2558 generic.go:356] "Generic (PLEG): container finished" podID="cb151e18-2bf5-4098-a956-829f7bbf9e4e" containerID="051762ffdd3576741ecbabb9cd4aaad7672d6becfb4d68eacc892acc8469cb68" exitCode=0 Jul 29 00:36:19.927514 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:19.927198 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c527k" event={"ID":"cb151e18-2bf5-4098-a956-829f7bbf9e4e","Type":"ContainerDied","Data":"051762ffdd3576741ecbabb9cd4aaad7672d6becfb4d68eacc892acc8469cb68"} Jul 29 00:36:19.927514 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:19.927260 2558 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 29 00:36:20.054071 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:20.054043 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:36:20.783994 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:20.783758 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:20.784176 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:20.784096 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:36:21.783721 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:21.783689 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:21.784111 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:21.783820 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2kc2k" podUID="c56793a6-2612-4e39-84fd-5c357c49066a" Jul 29 00:36:21.784111 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:21.783872 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:21.784111 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:21.784008 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgvr7" podUID="a00c6eb4-e6cb-4486-949f-f4f4a7ba538c" Jul 29 00:36:22.470579 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:22.470546 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:22.470729 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:22.470652 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 00:36:22.470729 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:22.470707 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret podName:a50c1131-277e-4dd2-9fdd-dfd5d5aabdee nodeName:}" failed. No retries permitted until 2026-07-29 00:36:54.470689502 +0000 UTC m=+66.129915373 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret") pod "global-pull-secret-syncer-jghtb" (UID: "a50c1131-277e-4dd2-9fdd-dfd5d5aabdee") : object "kube-system"/"original-pull-secret" not registered Jul 29 00:36:22.571390 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:22.571357 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj8ps\" (UniqueName: \"kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps\") pod \"network-check-target-2kc2k\" (UID: \"c56793a6-2612-4e39-84fd-5c357c49066a\") " pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:22.571560 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:22.571426 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs\") pod \"network-metrics-daemon-mgvr7\" (UID: \"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c\") " pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:22.571560 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:22.571528 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 00:36:22.571560 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:22.571539 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 00:36:22.571560 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:22.571548 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 00:36:22.571560 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:22.571557 2558 projected.go:196] Error preparing data for projected volume kube-api-access-wj8ps for pod openshift-network-diagnostics/network-check-target-2kc2k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 00:36:22.571794 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:22.571602 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs podName:a00c6eb4-e6cb-4486-949f-f4f4a7ba538c nodeName:}" failed. No retries permitted until 2026-07-29 00:36:54.57158282 +0000 UTC m=+66.230808704 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs") pod "network-metrics-daemon-mgvr7" (UID: "a00c6eb4-e6cb-4486-949f-f4f4a7ba538c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 00:36:22.571794 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:22.571620 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps podName:c56793a6-2612-4e39-84fd-5c357c49066a nodeName:}" failed. No retries permitted until 2026-07-29 00:36:54.571610841 +0000 UTC m=+66.230836718 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wj8ps" (UniqueName: "kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps") pod "network-check-target-2kc2k" (UID: "c56793a6-2612-4e39-84fd-5c357c49066a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 00:36:22.783488 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:22.783409 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:22.783637 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:22.783536 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jghtb" podUID="a50c1131-277e-4dd2-9fdd-dfd5d5aabdee" Jul 29 00:36:23.166526 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.166502 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-11.ec2.internal" event="NodeReady" Jul 29 00:36:23.167203 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.166654 2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 29 00:36:23.200282 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.200122 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77879686d6-qmm2n"] Jul 29 00:36:23.203037 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.203012 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.203504 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.203481 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-mf9ww"] Jul 29 00:36:23.205298 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.205205 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 29 00:36:23.205298 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.205236 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 29 00:36:23.205298 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.205241 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:23.205298 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.205256 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 29 00:36:23.205627 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.205598 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f4ctm\"" Jul 29 00:36:23.207217 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.207197 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 29 00:36:23.207450 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.207428 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2fjj8\"" Jul 29 00:36:23.207543 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.207516 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 29 00:36:23.207601 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.207563 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 29 00:36:23.207770 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.207751 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 29 00:36:23.212062 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.212044 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 29 00:36:23.217524 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.216272 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 29 00:36:23.220236 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.219341 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd"] Jul 29 00:36:23.221483 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.221334 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb"] Jul 29 00:36:23.221483 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.221406 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" Jul 29 00:36:23.223364 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.223277 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 29 00:36:23.223625 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.223611 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hztdg\"" Jul 29 00:36:23.224213 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.224195 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 29 00:36:23.224294 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.224195 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 29 00:36:23.224596 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.224423 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 29 00:36:23.224713 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.224695 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z"] Jul 29 00:36:23.224804 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.224790 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" Jul 29 00:36:23.226514 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.226496 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-7k7kp"] Jul 29 00:36:23.226647 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.226630 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" Jul 29 00:36:23.227669 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.227649 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 29 00:36:23.227733 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.227689 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-trzjw\"" Jul 29 00:36:23.227733 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.227696 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 29 00:36:23.227830 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.227738 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 29 00:36:23.228462 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.228447 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77879686d6-qmm2n"] Jul 29 00:36:23.228624 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.228604 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hzj7z"] Jul 29 00:36:23.229103 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.228543 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7k7kp" Jul 29 00:36:23.229103 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.228624 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 29 00:36:23.229103 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.228530 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 29 00:36:23.229103 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.228641 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2xjjq\"" Jul 29 00:36:23.230193 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.230173 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 29 00:36:23.230566 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.230549 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-mf9ww"] Jul 29 00:36:23.230724 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.230584 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6g74f"] Jul 29 00:36:23.230724 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.230676 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:23.231033 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.230985 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 29 00:36:23.231234 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.231218 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-wqr87\"" Jul 29 00:36:23.231359 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.231343 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 29 00:36:23.232452 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.232432 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6g74f" Jul 29 00:36:23.232550 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.232482 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 29 00:36:23.232606 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.232582 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 29 00:36:23.232653 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.232604 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2nxv8\"" Jul 29 00:36:23.234827 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.234810 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5x4gq\"" Jul 29 00:36:23.235194 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.235170 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 29 00:36:23.235460 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.235244 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 29 00:36:23.235695 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.235679 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 29 00:36:23.235949 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.235937 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-7k7kp"] Jul 29 00:36:23.236926 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.236911 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb"] Jul 29 00:36:23.237712 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.237689 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z"] Jul 29 00:36:23.241660 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.241630 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hzj7z"] Jul 29 00:36:23.242474 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.242441 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd"] Jul 29 00:36:23.249955 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.249935 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6g74f"] Jul 29 00:36:23.377093 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377008 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4c187d4-ebdd-4a5c-9d5b-a6774464084d-trusted-ca\") pod \"console-operator-584dcb6b54-mf9ww\" (UID: \"d4c187d4-ebdd-4a5c-9d5b-a6774464084d\") " pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:23.377093 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377052 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-certificates\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.377093 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377085 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93783ee9-0737-44f5-a9e3-51438955f859-tmp-dir\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:23.377339 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377130 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea6878e6-f9c1-4f52-8387-2c2c38d7084f-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb\" (UID: \"ea6878e6-f9c1-4f52-8387-2c2c38d7084f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" Jul 29 00:36:23.377339 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377164 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-bound-sa-token\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.377339 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377189 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4c187d4-ebdd-4a5c-9d5b-a6774464084d-serving-cert\") pod \"console-operator-584dcb6b54-mf9ww\" (UID: \"d4c187d4-ebdd-4a5c-9d5b-a6774464084d\") " pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:23.377339 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377211 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/711fe493-7a2b-41ca-b22e-a388bd42275a-serving-cert\") pod \"service-ca-operator-5d6c75f84f-2lrbd\" (UID: \"711fe493-7a2b-41ca-b22e-a388bd42275a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" Jul 29 00:36:23.377339 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377236 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4c187d4-ebdd-4a5c-9d5b-a6774464084d-config\") pod \"console-operator-584dcb6b54-mf9ww\" (UID: \"d4c187d4-ebdd-4a5c-9d5b-a6774464084d\") " pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:23.377339 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377257 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:23.377339 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377285 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5hxk\" (UniqueName: \"kubernetes.io/projected/96721a98-a363-4b5f-9c86-7539b6aab4c3-kube-api-access-l5hxk\") pod \"network-check-source-7d88cb55d8-7k7kp\" (UID: \"96721a98-a363-4b5f-9c86-7539b6aab4c3\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7k7kp" Jul 29 00:36:23.377662 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377353 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96wlb\" (UniqueName: \"kubernetes.io/projected/93783ee9-0737-44f5-a9e3-51438955f859-kube-api-access-96wlb\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:23.377662 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377392 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rzc9z\" (UID: \"a6663940-3227-4b6c-9569-69678c713609\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" Jul 29 00:36:23.377662 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377419 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-ca-trust-extracted\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.377662 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377449 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqh5h\" (UniqueName: \"kubernetes.io/projected/711fe493-7a2b-41ca-b22e-a388bd42275a-kube-api-access-dqh5h\") pod \"service-ca-operator-5d6c75f84f-2lrbd\" (UID: \"711fe493-7a2b-41ca-b22e-a388bd42275a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" Jul 29 00:36:23.377662 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377474 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a6663940-3227-4b6c-9569-69678c713609-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-rzc9z\" (UID: \"a6663940-3227-4b6c-9569-69678c713609\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" Jul 29 00:36:23.377662 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377520 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.377662 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377559 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-installation-pull-secrets\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.377662 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377611 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x458q\" (UniqueName: \"kubernetes.io/projected/1939d4b2-0a14-4f02-aa47-98e04683a8dd-kube-api-access-x458q\") pod \"ingress-canary-6g74f\" (UID: \"1939d4b2-0a14-4f02-aa47-98e04683a8dd\") " pod="openshift-ingress-canary/ingress-canary-6g74f" Jul 29 00:36:23.377662 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377633 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/711fe493-7a2b-41ca-b22e-a388bd42275a-config\") pod \"service-ca-operator-5d6c75f84f-2lrbd\" (UID: \"711fe493-7a2b-41ca-b22e-a388bd42275a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" Jul 29 00:36:23.378039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377695 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26dmp\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-kube-api-access-26dmp\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.378039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377725 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-image-registry-private-configuration\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.378039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377758 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc4kq\" (UniqueName: \"kubernetes.io/projected/ea6878e6-f9c1-4f52-8387-2c2c38d7084f-kube-api-access-zc4kq\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb\" (UID: \"ea6878e6-f9c1-4f52-8387-2c2c38d7084f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" Jul 29 00:36:23.378039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377786 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4j2c\" (UniqueName: \"kubernetes.io/projected/d4c187d4-ebdd-4a5c-9d5b-a6774464084d-kube-api-access-b4j2c\") pod \"console-operator-584dcb6b54-mf9ww\" (UID: \"d4c187d4-ebdd-4a5c-9d5b-a6774464084d\") " pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:23.378039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377814 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1939d4b2-0a14-4f02-aa47-98e04683a8dd-cert\") pod \"ingress-canary-6g74f\" (UID: \"1939d4b2-0a14-4f02-aa47-98e04683a8dd\") " pod="openshift-ingress-canary/ingress-canary-6g74f" Jul 29 00:36:23.378039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377866 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea6878e6-f9c1-4f52-8387-2c2c38d7084f-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb\" (UID: \"ea6878e6-f9c1-4f52-8387-2c2c38d7084f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" Jul 29 00:36:23.378039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377893 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/93783ee9-0737-44f5-a9e3-51438955f859-config-volume\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:23.378039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.377928 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-trusted-ca\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.389445 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.389414 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cjfg6_703966f4-444e-4b3b-854d-3a7b67259320/konnectivity-agent/0.log" limit=1024 Jul 29 00:36:23.476510 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.476476 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cjfg6_703966f4-444e-4b3b-854d-3a7b67259320/konnectivity-agent/0.log" limit=1024 Jul 29 00:36:23.478273 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478249 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/93783ee9-0737-44f5-a9e3-51438955f859-config-volume\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:23.478430 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478281 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-trusted-ca\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.478430 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478327 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4c187d4-ebdd-4a5c-9d5b-a6774464084d-trusted-ca\") pod \"console-operator-584dcb6b54-mf9ww\" (UID: \"d4c187d4-ebdd-4a5c-9d5b-a6774464084d\") " pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:23.478430 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478356 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-certificates\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.478575 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478470 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93783ee9-0737-44f5-a9e3-51438955f859-tmp-dir\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:23.478575 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478529 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea6878e6-f9c1-4f52-8387-2c2c38d7084f-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb\" (UID: \"ea6878e6-f9c1-4f52-8387-2c2c38d7084f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" Jul 29 00:36:23.478666 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478572 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-bound-sa-token\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.478666 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478603 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4c187d4-ebdd-4a5c-9d5b-a6774464084d-serving-cert\") pod \"console-operator-584dcb6b54-mf9ww\" (UID: \"d4c187d4-ebdd-4a5c-9d5b-a6774464084d\") " pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:23.478666 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478629 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/711fe493-7a2b-41ca-b22e-a388bd42275a-serving-cert\") pod \"service-ca-operator-5d6c75f84f-2lrbd\" (UID: \"711fe493-7a2b-41ca-b22e-a388bd42275a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" Jul 29 00:36:23.478666 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478656 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4c187d4-ebdd-4a5c-9d5b-a6774464084d-config\") pod \"console-operator-584dcb6b54-mf9ww\" (UID: \"d4c187d4-ebdd-4a5c-9d5b-a6774464084d\") " pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:23.478849 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478679 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:23.478849 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478708 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5hxk\" (UniqueName: \"kubernetes.io/projected/96721a98-a363-4b5f-9c86-7539b6aab4c3-kube-api-access-l5hxk\") pod \"network-check-source-7d88cb55d8-7k7kp\" (UID: \"96721a98-a363-4b5f-9c86-7539b6aab4c3\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7k7kp" Jul 29 00:36:23.478849 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478731 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96wlb\" (UniqueName: \"kubernetes.io/projected/93783ee9-0737-44f5-a9e3-51438955f859-kube-api-access-96wlb\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:23.478849 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478759 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rzc9z\" (UID: \"a6663940-3227-4b6c-9569-69678c713609\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" Jul 29 00:36:23.478849 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478776 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93783ee9-0737-44f5-a9e3-51438955f859-tmp-dir\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:23.478849 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478784 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-ca-trust-extracted\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.479077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478847 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqh5h\" (UniqueName: \"kubernetes.io/projected/711fe493-7a2b-41ca-b22e-a388bd42275a-kube-api-access-dqh5h\") pod \"service-ca-operator-5d6c75f84f-2lrbd\" (UID: \"711fe493-7a2b-41ca-b22e-a388bd42275a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" Jul 29 00:36:23.479077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478881 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a6663940-3227-4b6c-9569-69678c713609-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-rzc9z\" (UID: \"a6663940-3227-4b6c-9569-69678c713609\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" Jul 29 00:36:23.479077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478910 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.479077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478935 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-installation-pull-secrets\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.479077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478979 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x458q\" (UniqueName: \"kubernetes.io/projected/1939d4b2-0a14-4f02-aa47-98e04683a8dd-kube-api-access-x458q\") pod \"ingress-canary-6g74f\" (UID: \"1939d4b2-0a14-4f02-aa47-98e04683a8dd\") " pod="openshift-ingress-canary/ingress-canary-6g74f" Jul 29 00:36:23.479077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478987 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-certificates\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.479077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.479007 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/711fe493-7a2b-41ca-b22e-a388bd42275a-config\") pod \"service-ca-operator-5d6c75f84f-2lrbd\" (UID: \"711fe493-7a2b-41ca-b22e-a388bd42275a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" Jul 29 00:36:23.479077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.479044 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26dmp\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-kube-api-access-26dmp\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.479077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.479063 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-ca-trust-extracted\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.479077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.479069 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-image-registry-private-configuration\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.479518 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.479112 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc4kq\" (UniqueName: \"kubernetes.io/projected/ea6878e6-f9c1-4f52-8387-2c2c38d7084f-kube-api-access-zc4kq\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb\" (UID: \"ea6878e6-f9c1-4f52-8387-2c2c38d7084f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" Jul 29 00:36:23.479518 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.479146 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4j2c\" (UniqueName: \"kubernetes.io/projected/d4c187d4-ebdd-4a5c-9d5b-a6774464084d-kube-api-access-b4j2c\") pod \"console-operator-584dcb6b54-mf9ww\" (UID: \"d4c187d4-ebdd-4a5c-9d5b-a6774464084d\") " pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:23.479518 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:23.479152 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 00:36:23.479518 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.479174 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1939d4b2-0a14-4f02-aa47-98e04683a8dd-cert\") pod \"ingress-canary-6g74f\" (UID: \"1939d4b2-0a14-4f02-aa47-98e04683a8dd\") " pod="openshift-ingress-canary/ingress-canary-6g74f" Jul 29 00:36:23.479518 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:23.479228 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls podName:93783ee9-0737-44f5-a9e3-51438955f859 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:23.97920822 +0000 UTC m=+35.638434112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls") pod "dns-default-hzj7z" (UID: "93783ee9-0737-44f5-a9e3-51438955f859") : secret "dns-default-metrics-tls" not found Jul 29 00:36:23.479518 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.479250 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea6878e6-f9c1-4f52-8387-2c2c38d7084f-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb\" (UID: \"ea6878e6-f9c1-4f52-8387-2c2c38d7084f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" Jul 29 00:36:23.479518 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.479354 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4c187d4-ebdd-4a5c-9d5b-a6774464084d-trusted-ca\") pod \"console-operator-584dcb6b54-mf9ww\" (UID: \"d4c187d4-ebdd-4a5c-9d5b-a6774464084d\") " pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:23.479850 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.479595 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4c187d4-ebdd-4a5c-9d5b-a6774464084d-config\") pod \"console-operator-584dcb6b54-mf9ww\" (UID: \"d4c187d4-ebdd-4a5c-9d5b-a6774464084d\") " pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:23.479850 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:23.479774 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 29 00:36:23.479850 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.478979 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/93783ee9-0737-44f5-a9e3-51438955f859-config-volume\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:23.479850 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.479789 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea6878e6-f9c1-4f52-8387-2c2c38d7084f-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb\" (UID: \"ea6878e6-f9c1-4f52-8387-2c2c38d7084f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" Jul 29 00:36:23.480033 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:23.479881 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert podName:a6663940-3227-4b6c-9569-69678c713609 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:23.979853607 +0000 UTC m=+35.639079501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rzc9z" (UID: "a6663940-3227-4b6c-9569-69678c713609") : secret "networking-console-plugin-cert" not found Jul 29 00:36:23.480033 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:23.479979 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 00:36:23.480033 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:23.479992 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77879686d6-qmm2n: secret "image-registry-tls" not found Jul 29 00:36:23.480033 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:23.480036 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls podName:2fa28369-78d4-43b6-bba0-1ea67ef2a6a6 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:23.980023602 +0000 UTC m=+35.639249476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls") pod "image-registry-77879686d6-qmm2n" (UID: "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6") : secret "image-registry-tls" not found Jul 29 00:36:23.480764 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.480741 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/711fe493-7a2b-41ca-b22e-a388bd42275a-config\") pod \"service-ca-operator-5d6c75f84f-2lrbd\" (UID: \"711fe493-7a2b-41ca-b22e-a388bd42275a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" Jul 29 00:36:23.480842 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.480799 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a6663940-3227-4b6c-9569-69678c713609-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-rzc9z\" (UID: \"a6663940-3227-4b6c-9569-69678c713609\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" Jul 29 00:36:23.481547 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.481520 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-trusted-ca\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.483831 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.483803 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/711fe493-7a2b-41ca-b22e-a388bd42275a-serving-cert\") pod \"service-ca-operator-5d6c75f84f-2lrbd\" (UID: \"711fe493-7a2b-41ca-b22e-a388bd42275a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" Jul 29 00:36:23.483927 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.483850 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-image-registry-private-configuration\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.483927 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.483849 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4c187d4-ebdd-4a5c-9d5b-a6774464084d-serving-cert\") pod \"console-operator-584dcb6b54-mf9ww\" (UID: \"d4c187d4-ebdd-4a5c-9d5b-a6774464084d\") " pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:23.483927 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.483912 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1939d4b2-0a14-4f02-aa47-98e04683a8dd-cert\") pod \"ingress-canary-6g74f\" (UID: \"1939d4b2-0a14-4f02-aa47-98e04683a8dd\") " pod="openshift-ingress-canary/ingress-canary-6g74f" Jul 29 00:36:23.484695 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.484673 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea6878e6-f9c1-4f52-8387-2c2c38d7084f-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb\" (UID: \"ea6878e6-f9c1-4f52-8387-2c2c38d7084f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" Jul 29 00:36:23.485192 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.485166 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-installation-pull-secrets\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.488543 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.488497 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-bound-sa-token\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.488704 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.488677 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqh5h\" (UniqueName: \"kubernetes.io/projected/711fe493-7a2b-41ca-b22e-a388bd42275a-kube-api-access-dqh5h\") pod \"service-ca-operator-5d6c75f84f-2lrbd\" (UID: \"711fe493-7a2b-41ca-b22e-a388bd42275a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" Jul 29 00:36:23.488770 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.488683 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5hxk\" (UniqueName: \"kubernetes.io/projected/96721a98-a363-4b5f-9c86-7539b6aab4c3-kube-api-access-l5hxk\") pod \"network-check-source-7d88cb55d8-7k7kp\" (UID: \"96721a98-a363-4b5f-9c86-7539b6aab4c3\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7k7kp" Jul 29 00:36:23.488911 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.488892 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96wlb\" (UniqueName: \"kubernetes.io/projected/93783ee9-0737-44f5-a9e3-51438955f859-kube-api-access-96wlb\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:23.489208 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.489187 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x458q\" (UniqueName: \"kubernetes.io/projected/1939d4b2-0a14-4f02-aa47-98e04683a8dd-kube-api-access-x458q\") pod \"ingress-canary-6g74f\" (UID: \"1939d4b2-0a14-4f02-aa47-98e04683a8dd\") " pod="openshift-ingress-canary/ingress-canary-6g74f" Jul 29 00:36:23.490233 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.490195 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4j2c\" (UniqueName: \"kubernetes.io/projected/d4c187d4-ebdd-4a5c-9d5b-a6774464084d-kube-api-access-b4j2c\") pod \"console-operator-584dcb6b54-mf9ww\" (UID: \"d4c187d4-ebdd-4a5c-9d5b-a6774464084d\") " pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:23.490463 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.490439 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc4kq\" (UniqueName: \"kubernetes.io/projected/ea6878e6-f9c1-4f52-8387-2c2c38d7084f-kube-api-access-zc4kq\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb\" (UID: \"ea6878e6-f9c1-4f52-8387-2c2c38d7084f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" Jul 29 00:36:23.490561 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.490528 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26dmp\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-kube-api-access-26dmp\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.524148 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.524114 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:23.535445 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.535427 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" Jul 29 00:36:23.543030 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.543011 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" Jul 29 00:36:23.559009 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.558988 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7k7kp" Jul 29 00:36:23.571959 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.571944 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6g74f" Jul 29 00:36:23.783023 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.782919 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:23.783273 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.783117 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:23.785478 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.785455 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 00:36:23.785607 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.785494 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s4b5n\"" Jul 29 00:36:23.785607 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.785464 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-44db9\"" Jul 29 00:36:23.984692 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.984577 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:23.985950 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:23.984735 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 00:36:23.985950 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:23.984819 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls podName:93783ee9-0737-44f5-a9e3-51438955f859 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:24.98479691 +0000 UTC m=+36.644022784 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls") pod "dns-default-hzj7z" (UID: "93783ee9-0737-44f5-a9e3-51438955f859") : secret "dns-default-metrics-tls" not found Jul 29 00:36:23.985950 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.984632 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rzc9z\" (UID: \"a6663940-3227-4b6c-9569-69678c713609\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" Jul 29 00:36:23.985950 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:23.985362 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:23.985950 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:23.985541 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 00:36:23.985950 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:23.985561 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77879686d6-qmm2n: secret "image-registry-tls" not found Jul 29 00:36:23.985950 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:23.985621 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls podName:2fa28369-78d4-43b6-bba0-1ea67ef2a6a6 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:24.985601561 +0000 UTC m=+36.644827434 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls") pod "image-registry-77879686d6-qmm2n" (UID: "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6") : secret "image-registry-tls" not found Jul 29 00:36:23.985950 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:23.985704 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 29 00:36:23.985950 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:23.985746 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert podName:a6663940-3227-4b6c-9569-69678c713609 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:24.985734366 +0000 UTC m=+36.644960238 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rzc9z" (UID: "a6663940-3227-4b6c-9569-69678c713609") : secret "networking-console-plugin-cert" not found Jul 29 00:36:24.172007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.171969 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd"] Jul 29 00:36:24.178204 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:24.178177 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod711fe493_7a2b_41ca_b22e_a388bd42275a.slice/crio-175e336343f05896ec720ee9c61a10078448f367419083f064765436bacec0be WatchSource:0}: Error finding container 175e336343f05896ec720ee9c61a10078448f367419083f064765436bacec0be: Status 404 returned error can't find the container with id 175e336343f05896ec720ee9c61a10078448f367419083f064765436bacec0be Jul 29 00:36:24.187359 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.187337 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-7k7kp"] Jul 29 00:36:24.189874 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.189850 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-mf9ww"] Jul 29 00:36:24.193355 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:24.193327 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96721a98_a363_4b5f_9c86_7539b6aab4c3.slice/crio-1045cd32cfc9b83f85b77e81f72ff7268ffe0c0f90df43bb11380d0cc236cdfb WatchSource:0}: Error finding container 1045cd32cfc9b83f85b77e81f72ff7268ffe0c0f90df43bb11380d0cc236cdfb: Status 404 returned error can't find the container with id 1045cd32cfc9b83f85b77e81f72ff7268ffe0c0f90df43bb11380d0cc236cdfb Jul 29 00:36:24.193720 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:24.193698 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4c187d4_ebdd_4a5c_9d5b_a6774464084d.slice/crio-479eccf31fc13f6a341bac61025e87825914002376ced54064773977d08b4704 WatchSource:0}: Error finding container 479eccf31fc13f6a341bac61025e87825914002376ced54064773977d08b4704: Status 404 returned error can't find the container with id 479eccf31fc13f6a341bac61025e87825914002376ced54064773977d08b4704 Jul 29 00:36:24.199258 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.199236 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb"] Jul 29 00:36:24.202269 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.202084 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6g74f"] Jul 29 00:36:24.203269 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:24.203242 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea6878e6_f9c1_4f52_8387_2c2c38d7084f.slice/crio-2436d6ec04de9b8ad02cc9d1256c605ab57dd683d1881123cdf01256cb25a024 WatchSource:0}: Error finding container 2436d6ec04de9b8ad02cc9d1256c605ab57dd683d1881123cdf01256cb25a024: Status 404 returned error can't find the container with id 2436d6ec04de9b8ad02cc9d1256c605ab57dd683d1881123cdf01256cb25a024 Jul 29 00:36:24.212039 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:24.212021 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1939d4b2_0a14_4f02_aa47_98e04683a8dd.slice/crio-1f6a3f27b93c3f3b72d8d5e55c44b2f0c5583939b7190c860760f710c3a03c29 WatchSource:0}: Error finding container 1f6a3f27b93c3f3b72d8d5e55c44b2f0c5583939b7190c860760f710c3a03c29: Status 404 returned error can't find the container with id 1f6a3f27b93c3f3b72d8d5e55c44b2f0c5583939b7190c860760f710c3a03c29 Jul 29 00:36:24.785637 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.785601 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:24.787858 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.787825 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 00:36:24.941412 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.941380 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6g74f" event={"ID":"1939d4b2-0a14-4f02-aa47-98e04683a8dd","Type":"ContainerStarted","Data":"1f6a3f27b93c3f3b72d8d5e55c44b2f0c5583939b7190c860760f710c3a03c29"} Jul 29 00:36:24.942562 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.942538 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" event={"ID":"711fe493-7a2b-41ca-b22e-a388bd42275a","Type":"ContainerStarted","Data":"175e336343f05896ec720ee9c61a10078448f367419083f064765436bacec0be"} Jul 29 00:36:24.945785 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.945745 2558 generic.go:356] "Generic (PLEG): container finished" podID="cb151e18-2bf5-4098-a956-829f7bbf9e4e" containerID="8d02e9367a0f111d04f1b2c4d5c8795f0cc516c65cb6d72c154ca889ccdef9ea" exitCode=0 Jul 29 00:36:24.945916 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.945829 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c527k" event={"ID":"cb151e18-2bf5-4098-a956-829f7bbf9e4e","Type":"ContainerDied","Data":"8d02e9367a0f111d04f1b2c4d5c8795f0cc516c65cb6d72c154ca889ccdef9ea"} Jul 29 00:36:24.947622 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.947572 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" event={"ID":"ea6878e6-f9c1-4f52-8387-2c2c38d7084f","Type":"ContainerStarted","Data":"2436d6ec04de9b8ad02cc9d1256c605ab57dd683d1881123cdf01256cb25a024"} Jul 29 00:36:24.951027 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.950998 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" event={"ID":"d4c187d4-ebdd-4a5c-9d5b-a6774464084d","Type":"ContainerStarted","Data":"479eccf31fc13f6a341bac61025e87825914002376ced54064773977d08b4704"} Jul 29 00:36:24.957703 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.954675 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7k7kp" event={"ID":"96721a98-a363-4b5f-9c86-7539b6aab4c3","Type":"ContainerStarted","Data":"1045cd32cfc9b83f85b77e81f72ff7268ffe0c0f90df43bb11380d0cc236cdfb"} Jul 29 00:36:24.996072 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.995359 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:24.996072 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.995492 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:24.996072 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:24.995525 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rzc9z\" (UID: \"a6663940-3227-4b6c-9569-69678c713609\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" Jul 29 00:36:24.996072 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:24.995649 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 29 00:36:24.996072 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:24.995704 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert podName:a6663940-3227-4b6c-9569-69678c713609 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:26.995685997 +0000 UTC m=+38.654911873 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rzc9z" (UID: "a6663940-3227-4b6c-9569-69678c713609") : secret "networking-console-plugin-cert" not found Jul 29 00:36:24.996072 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:24.995964 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 00:36:24.996072 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:24.996014 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls podName:93783ee9-0737-44f5-a9e3-51438955f859 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:26.995999038 +0000 UTC m=+38.655224919 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls") pod "dns-default-hzj7z" (UID: "93783ee9-0737-44f5-a9e3-51438955f859") : secret "dns-default-metrics-tls" not found Jul 29 00:36:24.996072 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:24.996079 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 00:36:24.996506 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:24.996090 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77879686d6-qmm2n: secret "image-registry-tls" not found Jul 29 00:36:24.996506 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:24.996125 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls podName:2fa28369-78d4-43b6-bba0-1ea67ef2a6a6 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:26.996114022 +0000 UTC m=+38.655339908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls") pod "image-registry-77879686d6-qmm2n" (UID: "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6") : secret "image-registry-tls" not found Jul 29 00:36:25.965258 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:25.965219 2558 generic.go:356] "Generic (PLEG): container finished" podID="cb151e18-2bf5-4098-a956-829f7bbf9e4e" containerID="f9ea87b3ccdd1710428e5ed470c1e57ec28a778e601032664266e0e0bc363f7e" exitCode=0 Jul 29 00:36:25.965811 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:25.965325 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c527k" event={"ID":"cb151e18-2bf5-4098-a956-829f7bbf9e4e","Type":"ContainerDied","Data":"f9ea87b3ccdd1710428e5ed470c1e57ec28a778e601032664266e0e0bc363f7e"} Jul 29 00:36:26.971833 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:26.971627 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c527k" event={"ID":"cb151e18-2bf5-4098-a956-829f7bbf9e4e","Type":"ContainerStarted","Data":"b93d7c91f7c5b80e142679c3b70045170db06c0f1ba7b29a72541db888f06d02"} Jul 29 00:36:26.993732 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:26.993680 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-c527k" podStartSLOduration=7.669373741 podStartE2EDuration="37.993663295s" totalImagesPullingTime="30.324289554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:35:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:26.993183246 +0000 UTC m=+38.652409139" watchObservedRunningTime="2026-07-29 00:36:26.993663295 +0000 UTC m=+38.652889189" Jul 29 00:36:27.015445 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:27.015413 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:27.015596 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:27.015460 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rzc9z\" (UID: \"a6663940-3227-4b6c-9569-69678c713609\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" Jul 29 00:36:27.015596 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:27.015497 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:27.015682 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:27.015593 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 00:36:27.015682 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:27.015621 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 29 00:36:27.015682 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:27.015600 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 00:36:27.015682 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:27.015660 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls podName:93783ee9-0737-44f5-a9e3-51438955f859 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:31.015640282 +0000 UTC m=+42.674866188 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls") pod "dns-default-hzj7z" (UID: "93783ee9-0737-44f5-a9e3-51438955f859") : secret "dns-default-metrics-tls" not found Jul 29 00:36:27.015682 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:27.015662 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77879686d6-qmm2n: secret "image-registry-tls" not found Jul 29 00:36:27.015682 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:27.015680 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert podName:a6663940-3227-4b6c-9569-69678c713609 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:31.015669609 +0000 UTC m=+42.674895486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rzc9z" (UID: "a6663940-3227-4b6c-9569-69678c713609") : secret "networking-console-plugin-cert" not found Jul 29 00:36:27.015939 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:27.015778 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls podName:2fa28369-78d4-43b6-bba0-1ea67ef2a6a6 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:31.015756415 +0000 UTC m=+42.674982300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls") pod "image-registry-77879686d6-qmm2n" (UID: "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6") : secret "image-registry-tls" not found Jul 29 00:36:27.145785 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:27.145742 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cjfg6_703966f4-444e-4b3b-854d-3a7b67259320/konnectivity-agent/0.log" limit=1024 Jul 29 00:36:27.228990 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:27.228377 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cjfg6_703966f4-444e-4b3b-854d-3a7b67259320/konnectivity-agent/0.log" limit=1024 Jul 29 00:36:28.876800 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:28.876746 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cjfg6_703966f4-444e-4b3b-854d-3a7b67259320/konnectivity-agent/0.log" limit=1024 Jul 29 00:36:28.945988 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:28.945959 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cjfg6_703966f4-444e-4b3b-854d-3a7b67259320/konnectivity-agent/0.log" limit=1024 Jul 29 00:36:30.984272 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:30.984182 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6g74f" event={"ID":"1939d4b2-0a14-4f02-aa47-98e04683a8dd","Type":"ContainerStarted","Data":"68bbc573158691b6655ca014fef9aded5d56660ba06691245471e726d463b3a7"} Jul 29 00:36:30.986155 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:30.986127 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" event={"ID":"711fe493-7a2b-41ca-b22e-a388bd42275a","Type":"ContainerStarted","Data":"1f08f7d53403c9bc2fe39ebb4598e3b5a0d86dba9720905fdc5a3b87a955df12"} Jul 29 00:36:30.987778 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:30.987673 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" event={"ID":"ea6878e6-f9c1-4f52-8387-2c2c38d7084f","Type":"ContainerStarted","Data":"ca8d90ca22298924540df3e4c2d3bc6635f1f2cb80caa451f38a67dbd5104d2f"} Jul 29 00:36:30.989440 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:30.989419 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/0.log" Jul 29 00:36:30.989549 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:30.989460 2558 generic.go:356] "Generic (PLEG): container finished" podID="d4c187d4-ebdd-4a5c-9d5b-a6774464084d" containerID="ebf0a494a83df72664bcf4550a8d978267fcf0f09419eea283a886c9cfd5660f" exitCode=255 Jul 29 00:36:30.989549 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:30.989531 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" event={"ID":"d4c187d4-ebdd-4a5c-9d5b-a6774464084d","Type":"ContainerDied","Data":"ebf0a494a83df72664bcf4550a8d978267fcf0f09419eea283a886c9cfd5660f"} Jul 29 00:36:30.989740 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:30.989727 2558 scope.go:117] "RemoveContainer" containerID="ebf0a494a83df72664bcf4550a8d978267fcf0f09419eea283a886c9cfd5660f" Jul 29 00:36:30.991200 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:30.991173 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7k7kp" event={"ID":"96721a98-a363-4b5f-9c86-7539b6aab4c3","Type":"ContainerStarted","Data":"8d4ee1b80a8f744074c7c03d4339b9fb7700ff12ba38d4ad89c38ad3c9f0e269"} Jul 29 00:36:31.000109 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:31.000066 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6g74f" podStartSLOduration=1.6143906110000001 podStartE2EDuration="8.000050349s" totalImagesPullingTime="6.385659738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:30.998677885 +0000 UTC m=+42.657903778" watchObservedRunningTime="2026-07-29 00:36:31.000050349 +0000 UTC m=+42.659276269" Jul 29 00:36:31.013501 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:31.012741 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" podStartSLOduration=29.591573659 podStartE2EDuration="36.012721746s" totalImagesPullingTime="6.421148087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:35:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:31.012084336 +0000 UTC m=+42.671310230" watchObservedRunningTime="2026-07-29 00:36:31.012721746 +0000 UTC m=+42.671947640" Jul 29 00:36:31.043853 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:31.042636 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" podStartSLOduration=31.650500597 podStartE2EDuration="38.042611192s" totalImagesPullingTime="6.392110595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:35:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:31.041693872 +0000 UTC m=+42.700919768" watchObservedRunningTime="2026-07-29 00:36:31.042611192 +0000 UTC m=+42.701837092" Jul 29 00:36:31.043853 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:31.043534 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7k7kp" podStartSLOduration=29.640553524 podStartE2EDuration="36.043525546s" totalImagesPullingTime="6.402972022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:35:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:31.026552143 +0000 UTC m=+42.685778037" watchObservedRunningTime="2026-07-29 00:36:31.043525546 +0000 UTC m=+42.702751442" Jul 29 00:36:31.048923 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:31.048900 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:31.049120 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:31.049102 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rzc9z\" (UID: \"a6663940-3227-4b6c-9569-69678c713609\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" Jul 29 00:36:31.049248 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:31.049234 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:31.049527 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:31.049506 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 29 00:36:31.049706 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:31.049694 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert podName:a6663940-3227-4b6c-9569-69678c713609 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:39.049675083 +0000 UTC m=+50.708900958 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rzc9z" (UID: "a6663940-3227-4b6c-9569-69678c713609") : secret "networking-console-plugin-cert" not found Jul 29 00:36:31.050553 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:31.050175 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 00:36:31.050553 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:31.050244 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls podName:93783ee9-0737-44f5-a9e3-51438955f859 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:39.050218929 +0000 UTC m=+50.709444801 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls") pod "dns-default-hzj7z" (UID: "93783ee9-0737-44f5-a9e3-51438955f859") : secret "dns-default-metrics-tls" not found Jul 29 00:36:31.050714 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:31.050578 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 00:36:31.050714 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:31.050597 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77879686d6-qmm2n: secret "image-registry-tls" not found Jul 29 00:36:31.050714 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:31.050650 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls podName:2fa28369-78d4-43b6-bba0-1ea67ef2a6a6 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:39.050629008 +0000 UTC m=+50.709854882 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls") pod "image-registry-77879686d6-qmm2n" (UID: "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6") : secret "image-registry-tls" not found Jul 29 00:36:31.996738 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:31.996654 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 00:36:31.997152 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:31.997136 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/0.log" Jul 29 00:36:31.997202 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:31.997170 2558 generic.go:356] "Generic (PLEG): container finished" podID="d4c187d4-ebdd-4a5c-9d5b-a6774464084d" containerID="c21ded57b0cafe179a190b72d87c33b4b5938fa3c517b0ff5983b10b5b613456" exitCode=255 Jul 29 00:36:31.997294 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:31.997267 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" event={"ID":"d4c187d4-ebdd-4a5c-9d5b-a6774464084d","Type":"ContainerDied","Data":"c21ded57b0cafe179a190b72d87c33b4b5938fa3c517b0ff5983b10b5b613456"} Jul 29 00:36:31.997410 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:31.997390 2558 scope.go:117] "RemoveContainer" containerID="ebf0a494a83df72664bcf4550a8d978267fcf0f09419eea283a886c9cfd5660f" Jul 29 00:36:31.997966 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:31.997947 2558 scope.go:117] "RemoveContainer" containerID="c21ded57b0cafe179a190b72d87c33b4b5938fa3c517b0ff5983b10b5b613456" Jul 29 00:36:31.998120 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:31.998098 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-mf9ww_openshift-console-operator(d4c187d4-ebdd-4a5c-9d5b-a6774464084d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" podUID="d4c187d4-ebdd-4a5c-9d5b-a6774464084d" Jul 29 00:36:32.057619 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.057578 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cjfg6_703966f4-444e-4b3b-854d-3a7b67259320/konnectivity-agent/0.log" limit=1024 Jul 29 00:36:32.064168 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.064140 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-m7dmb"] Jul 29 00:36:32.096192 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.096169 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-m7dmb"] Jul 29 00:36:32.096346 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.096287 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7dmb" Jul 29 00:36:32.098445 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.098420 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 29 00:36:32.098562 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.098463 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 29 00:36:32.098562 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.098487 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-xdtvb\"" Jul 29 00:36:32.121228 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.121204 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cjfg6_703966f4-444e-4b3b-854d-3a7b67259320/konnectivity-agent/0.log" limit=1024 Jul 29 00:36:32.158352 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.158332 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trmfk\" (UniqueName: \"kubernetes.io/projected/dd5b2b10-6978-4787-9bec-74219be83e1f-kube-api-access-trmfk\") pod \"migrator-6865c9967-m7dmb\" (UID: \"dd5b2b10-6978-4787-9bec-74219be83e1f\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7dmb" Jul 29 00:36:32.179359 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.179333 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cjfg6_703966f4-444e-4b3b-854d-3a7b67259320/konnectivity-agent/0.log" limit=1024 Jul 29 00:36:32.259094 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.259023 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trmfk\" (UniqueName: \"kubernetes.io/projected/dd5b2b10-6978-4787-9bec-74219be83e1f-kube-api-access-trmfk\") pod \"migrator-6865c9967-m7dmb\" (UID: \"dd5b2b10-6978-4787-9bec-74219be83e1f\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7dmb" Jul 29 00:36:32.270531 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.270505 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trmfk\" (UniqueName: \"kubernetes.io/projected/dd5b2b10-6978-4787-9bec-74219be83e1f-kube-api-access-trmfk\") pod \"migrator-6865c9967-m7dmb\" (UID: \"dd5b2b10-6978-4787-9bec-74219be83e1f\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7dmb" Jul 29 00:36:32.393346 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.393293 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cjfg6_703966f4-444e-4b3b-854d-3a7b67259320/konnectivity-agent/0.log" limit=1024 Jul 29 00:36:32.405118 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.405094 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7dmb" Jul 29 00:36:32.543550 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:32.543474 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-m7dmb"] Jul 29 00:36:32.546293 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:32.546267 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd5b2b10_6978_4787_9bec_74219be83e1f.slice/crio-54768bbd287f6566dab997acc689878ac4ab947580a969925e4047590795eef4 WatchSource:0}: Error finding container 54768bbd287f6566dab997acc689878ac4ab947580a969925e4047590795eef4: Status 404 returned error can't find the container with id 54768bbd287f6566dab997acc689878ac4ab947580a969925e4047590795eef4 Jul 29 00:36:33.002127 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:33.002091 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7dmb" event={"ID":"dd5b2b10-6978-4787-9bec-74219be83e1f","Type":"ContainerStarted","Data":"54768bbd287f6566dab997acc689878ac4ab947580a969925e4047590795eef4"} Jul 29 00:36:33.003677 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:33.003655 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 00:36:33.003954 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:33.003940 2558 scope.go:117] "RemoveContainer" containerID="c21ded57b0cafe179a190b72d87c33b4b5938fa3c517b0ff5983b10b5b613456" Jul 29 00:36:33.004092 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:33.004076 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-mf9ww_openshift-console-operator(d4c187d4-ebdd-4a5c-9d5b-a6774464084d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" podUID="d4c187d4-ebdd-4a5c-9d5b-a6774464084d" Jul 29 00:36:33.524559 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:33.524519 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:33.524716 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:33.524593 2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:34.007397 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.007359 2558 scope.go:117] "RemoveContainer" containerID="c21ded57b0cafe179a190b72d87c33b4b5938fa3c517b0ff5983b10b5b613456" Jul 29 00:36:34.007834 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:34.007530 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-mf9ww_openshift-console-operator(d4c187d4-ebdd-4a5c-9d5b-a6774464084d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" podUID="d4c187d4-ebdd-4a5c-9d5b-a6774464084d" Jul 29 00:36:34.259269 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.259201 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-56nv9"] Jul 29 00:36:34.274766 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.274738 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-56nv9"] Jul 29 00:36:34.274899 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.274834 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-56nv9" Jul 29 00:36:34.276881 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.276854 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 29 00:36:34.276993 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.276863 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 29 00:36:34.277429 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.277410 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-7j997\"" Jul 29 00:36:34.277501 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.277413 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 29 00:36:34.277501 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.277452 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 29 00:36:34.375117 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.375088 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a1e8f9e5-2e50-496d-ab41-3e5b4bce506a-signing-cabundle\") pod \"service-ca-658d7b7c78-56nv9\" (UID: \"a1e8f9e5-2e50-496d-ab41-3e5b4bce506a\") " pod="openshift-service-ca/service-ca-658d7b7c78-56nv9" Jul 29 00:36:34.375254 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.375146 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a1e8f9e5-2e50-496d-ab41-3e5b4bce506a-signing-key\") pod \"service-ca-658d7b7c78-56nv9\" (UID: \"a1e8f9e5-2e50-496d-ab41-3e5b4bce506a\") " pod="openshift-service-ca/service-ca-658d7b7c78-56nv9" Jul 29 00:36:34.375254 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.375190 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb5vz\" (UniqueName: \"kubernetes.io/projected/a1e8f9e5-2e50-496d-ab41-3e5b4bce506a-kube-api-access-rb5vz\") pod \"service-ca-658d7b7c78-56nv9\" (UID: \"a1e8f9e5-2e50-496d-ab41-3e5b4bce506a\") " pod="openshift-service-ca/service-ca-658d7b7c78-56nv9" Jul 29 00:36:34.475777 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.475751 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a1e8f9e5-2e50-496d-ab41-3e5b4bce506a-signing-cabundle\") pod \"service-ca-658d7b7c78-56nv9\" (UID: \"a1e8f9e5-2e50-496d-ab41-3e5b4bce506a\") " pod="openshift-service-ca/service-ca-658d7b7c78-56nv9" Jul 29 00:36:34.475860 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.475816 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a1e8f9e5-2e50-496d-ab41-3e5b4bce506a-signing-key\") pod \"service-ca-658d7b7c78-56nv9\" (UID: \"a1e8f9e5-2e50-496d-ab41-3e5b4bce506a\") " pod="openshift-service-ca/service-ca-658d7b7c78-56nv9" Jul 29 00:36:34.475902 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.475866 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb5vz\" (UniqueName: \"kubernetes.io/projected/a1e8f9e5-2e50-496d-ab41-3e5b4bce506a-kube-api-access-rb5vz\") pod \"service-ca-658d7b7c78-56nv9\" (UID: \"a1e8f9e5-2e50-496d-ab41-3e5b4bce506a\") " pod="openshift-service-ca/service-ca-658d7b7c78-56nv9" Jul 29 00:36:34.476437 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.476415 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a1e8f9e5-2e50-496d-ab41-3e5b4bce506a-signing-cabundle\") pod \"service-ca-658d7b7c78-56nv9\" (UID: \"a1e8f9e5-2e50-496d-ab41-3e5b4bce506a\") " pod="openshift-service-ca/service-ca-658d7b7c78-56nv9" Jul 29 00:36:34.478223 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.478201 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a1e8f9e5-2e50-496d-ab41-3e5b4bce506a-signing-key\") pod \"service-ca-658d7b7c78-56nv9\" (UID: \"a1e8f9e5-2e50-496d-ab41-3e5b4bce506a\") " pod="openshift-service-ca/service-ca-658d7b7c78-56nv9" Jul 29 00:36:34.484236 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.484218 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb5vz\" (UniqueName: \"kubernetes.io/projected/a1e8f9e5-2e50-496d-ab41-3e5b4bce506a-kube-api-access-rb5vz\") pod \"service-ca-658d7b7c78-56nv9\" (UID: \"a1e8f9e5-2e50-496d-ab41-3e5b4bce506a\") " pod="openshift-service-ca/service-ca-658d7b7c78-56nv9" Jul 29 00:36:34.584215 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.584191 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-56nv9" Jul 29 00:36:34.739903 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:34.739880 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-56nv9"] Jul 29 00:36:34.741988 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:34.741964 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1e8f9e5_2e50_496d_ab41_3e5b4bce506a.slice/crio-b74353cd3ebe108cf6df7a264156b2f54dbb984bd04614e653cf5741c9879cc0 WatchSource:0}: Error finding container b74353cd3ebe108cf6df7a264156b2f54dbb984bd04614e653cf5741c9879cc0: Status 404 returned error can't find the container with id b74353cd3ebe108cf6df7a264156b2f54dbb984bd04614e653cf5741c9879cc0 Jul 29 00:36:35.011351 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:35.011295 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-56nv9" event={"ID":"a1e8f9e5-2e50-496d-ab41-3e5b4bce506a","Type":"ContainerStarted","Data":"6805354cf3a3444ae3a94f5c25a6586ec3c6be3f15e8aeccb010aa94886dd72e"} Jul 29 00:36:35.011775 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:35.011370 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-56nv9" event={"ID":"a1e8f9e5-2e50-496d-ab41-3e5b4bce506a","Type":"ContainerStarted","Data":"b74353cd3ebe108cf6df7a264156b2f54dbb984bd04614e653cf5741c9879cc0"} Jul 29 00:36:35.013075 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:35.013052 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7dmb" event={"ID":"dd5b2b10-6978-4787-9bec-74219be83e1f","Type":"ContainerStarted","Data":"d4e9f8b086cb8b6da03b0f637af4c36fd5212283c5a00bbc0ff9a75fc134957d"} Jul 29 00:36:35.013183 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:35.013092 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7dmb" event={"ID":"dd5b2b10-6978-4787-9bec-74219be83e1f","Type":"ContainerStarted","Data":"5646f6ef804de219127240ad5327d2688adbbc0d996f0b16850acef5f62297cb"} Jul 29 00:36:35.027340 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:35.027249 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-56nv9" podStartSLOduration=1.027234401 podStartE2EDuration="1.027234401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:35.026714187 +0000 UTC m=+46.685940079" watchObservedRunningTime="2026-07-29 00:36:35.027234401 +0000 UTC m=+46.686460294" Jul 29 00:36:35.044246 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:35.044205 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7dmb" podStartSLOduration=1.135109675 podStartE2EDuration="3.044195702s" totalImagesPullingTime="1.909086027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:35.043285632 +0000 UTC m=+46.702511524" watchObservedRunningTime="2026-07-29 00:36:35.044195702 +0000 UTC m=+46.703421616" Jul 29 00:36:39.116357 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:39.116297 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rzc9z\" (UID: \"a6663940-3227-4b6c-9569-69678c713609\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" Jul 29 00:36:39.116807 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:39.116373 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:39.116807 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:39.116441 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 29 00:36:39.116807 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:39.116490 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:39.116807 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:39.116511 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert podName:a6663940-3227-4b6c-9569-69678c713609 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:55.116493606 +0000 UTC m=+66.775719478 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rzc9z" (UID: "a6663940-3227-4b6c-9569-69678c713609") : secret "networking-console-plugin-cert" not found Jul 29 00:36:39.116807 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:39.116534 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 00:36:39.116807 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:39.116555 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77879686d6-qmm2n: secret "image-registry-tls" not found Jul 29 00:36:39.116807 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:39.116567 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 00:36:39.116807 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:39.116610 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls podName:93783ee9-0737-44f5-a9e3-51438955f859 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:55.116595625 +0000 UTC m=+66.775821503 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls") pod "dns-default-hzj7z" (UID: "93783ee9-0737-44f5-a9e3-51438955f859") : secret "dns-default-metrics-tls" not found Jul 29 00:36:39.116807 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:36:39.116630 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls podName:2fa28369-78d4-43b6-bba0-1ea67ef2a6a6 nodeName:}" failed. No retries permitted until 2026-07-29 00:36:55.116617675 +0000 UTC m=+66.775843549 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls") pod "image-registry-77879686d6-qmm2n" (UID: "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6") : secret "image-registry-tls" not found Jul 29 00:36:45.784122 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:45.784086 2558 scope.go:117] "RemoveContainer" containerID="c21ded57b0cafe179a190b72d87c33b4b5938fa3c517b0ff5983b10b5b613456" Jul 29 00:36:46.054216 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:46.054139 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 00:36:46.054376 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:46.054225 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" event={"ID":"d4c187d4-ebdd-4a5c-9d5b-a6774464084d","Type":"ContainerStarted","Data":"be65263ebaa22455f977bb4ddce63ec094f46240fd1cb0a1febb7daec93885a9"} Jul 29 00:36:46.054538 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:46.054520 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:46.072077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:46.072026 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" podStartSLOduration=50.670446586 podStartE2EDuration="57.07201011s" totalImagesPullingTime="6.401563524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:35:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:46.071033461 +0000 UTC m=+57.730259351" watchObservedRunningTime="2026-07-29 00:36:46.07201011 +0000 UTC m=+57.731236028" Jul 29 00:36:46.396508 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:46.396474 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-mf9ww" Jul 29 00:36:50.940382 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:50.940345 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mtt7s" Jul 29 00:36:54.545259 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.545221 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:54.547336 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.547302 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 00:36:54.557841 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.557814 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a50c1131-277e-4dd2-9fdd-dfd5d5aabdee-original-pull-secret\") pod \"global-pull-secret-syncer-jghtb\" (UID: \"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee\") " pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:54.646406 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.646373 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs\") pod \"network-metrics-daemon-mgvr7\" (UID: \"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c\") " pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:54.646550 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.646441 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj8ps\" (UniqueName: \"kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps\") pod \"network-check-target-2kc2k\" (UID: \"c56793a6-2612-4e39-84fd-5c357c49066a\") " pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:54.648816 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.648795 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 00:36:54.648947 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.648926 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj8ps\" (UniqueName: \"kubernetes.io/projected/c56793a6-2612-4e39-84fd-5c357c49066a-kube-api-access-wj8ps\") pod \"network-check-target-2kc2k\" (UID: \"c56793a6-2612-4e39-84fd-5c357c49066a\") " pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:54.659246 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.659224 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a00c6eb4-e6cb-4486-949f-f4f4a7ba538c-metrics-certs\") pod \"network-metrics-daemon-mgvr7\" (UID: \"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c\") " pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:54.696119 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.696093 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s4b5n\"" Jul 29 00:36:54.700776 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.700751 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-44db9\"" Jul 29 00:36:54.705073 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.705059 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgvr7" Jul 29 00:36:54.710286 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.710268 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:54.796753 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.796493 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jghtb" Jul 29 00:36:54.869144 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.869115 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mgvr7"] Jul 29 00:36:54.874869 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:54.874833 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda00c6eb4_e6cb_4486_949f_f4f4a7ba538c.slice/crio-a015c04c769c6510a4730fe7843bf8326ba67c2b326366b8c60bf4af08fa62e8 WatchSource:0}: Error finding container a015c04c769c6510a4730fe7843bf8326ba67c2b326366b8c60bf4af08fa62e8: Status 404 returned error can't find the container with id a015c04c769c6510a4730fe7843bf8326ba67c2b326366b8c60bf4af08fa62e8 Jul 29 00:36:54.893677 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.893650 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2kc2k"] Jul 29 00:36:54.897009 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:54.896985 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc56793a6_2612_4e39_84fd_5c357c49066a.slice/crio-397b9a37b36f2f643eccb7ad1315589d9ec6dfde03a5bbb620abc7dfee618c59 WatchSource:0}: Error finding container 397b9a37b36f2f643eccb7ad1315589d9ec6dfde03a5bbb620abc7dfee618c59: Status 404 returned error can't find the container with id 397b9a37b36f2f643eccb7ad1315589d9ec6dfde03a5bbb620abc7dfee618c59 Jul 29 00:36:54.952234 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:54.952205 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jghtb"] Jul 29 00:36:54.956297 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:54.956272 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda50c1131_277e_4dd2_9fdd_dfd5d5aabdee.slice/crio-1712448fc68833fdcc92088197c599826bc528d930cfe17c95e359c989fc3c7f WatchSource:0}: Error finding container 1712448fc68833fdcc92088197c599826bc528d930cfe17c95e359c989fc3c7f: Status 404 returned error can't find the container with id 1712448fc68833fdcc92088197c599826bc528d930cfe17c95e359c989fc3c7f Jul 29 00:36:55.081520 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.081492 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2kc2k" event={"ID":"c56793a6-2612-4e39-84fd-5c357c49066a","Type":"ContainerStarted","Data":"9deba6dc7078a494b717662386b83ed075eb333fb28300a09ca731159cf93188"} Jul 29 00:36:55.081659 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.081540 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2kc2k" event={"ID":"c56793a6-2612-4e39-84fd-5c357c49066a","Type":"ContainerStarted","Data":"397b9a37b36f2f643eccb7ad1315589d9ec6dfde03a5bbb620abc7dfee618c59"} Jul 29 00:36:55.081659 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.081587 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:36:55.082677 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.082654 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jghtb" event={"ID":"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee","Type":"ContainerStarted","Data":"1712448fc68833fdcc92088197c599826bc528d930cfe17c95e359c989fc3c7f"} Jul 29 00:36:55.083776 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.083759 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mgvr7" event={"ID":"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c","Type":"ContainerStarted","Data":"a015c04c769c6510a4730fe7843bf8326ba67c2b326366b8c60bf4af08fa62e8"} Jul 29 00:36:55.096842 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.096796 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2kc2k" podStartSLOduration=66.096781536 podStartE2EDuration="1m6.096781536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:35:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:55.096668937 +0000 UTC m=+66.755894829" watchObservedRunningTime="2026-07-29 00:36:55.096781536 +0000 UTC m=+66.756007430" Jul 29 00:36:55.150221 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.150195 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:55.150325 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.150230 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rzc9z\" (UID: \"a6663940-3227-4b6c-9569-69678c713609\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" Jul 29 00:36:55.150325 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.150252 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:55.152565 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.152542 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93783ee9-0737-44f5-a9e3-51438955f859-metrics-tls\") pod \"dns-default-hzj7z\" (UID: \"93783ee9-0737-44f5-a9e3-51438955f859\") " pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:55.152642 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.152588 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a6663940-3227-4b6c-9569-69678c713609-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rzc9z\" (UID: \"a6663940-3227-4b6c-9569-69678c713609\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" Jul 29 00:36:55.152642 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.152619 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls\") pod \"image-registry-77879686d6-qmm2n\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:55.318233 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.318203 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f4ctm\"" Jul 29 00:36:55.327149 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.327128 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:55.354539 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.354521 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2xjjq\"" Jul 29 00:36:55.363240 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.363217 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" Jul 29 00:36:55.368484 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.368460 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2nxv8\"" Jul 29 00:36:55.377041 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.377017 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hzj7z" Jul 29 00:36:55.487520 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.487492 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77879686d6-qmm2n"] Jul 29 00:36:55.490648 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:55.490615 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fa28369_78d4_43b6_bba0_1ea67ef2a6a6.slice/crio-3e0594451135d4814bf7e4052e45242a0bd0c13f16f5dc23371a754aea4250d4 WatchSource:0}: Error finding container 3e0594451135d4814bf7e4052e45242a0bd0c13f16f5dc23371a754aea4250d4: Status 404 returned error can't find the container with id 3e0594451135d4814bf7e4052e45242a0bd0c13f16f5dc23371a754aea4250d4 Jul 29 00:36:55.534342 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.534289 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z"] Jul 29 00:36:55.538750 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:55.538724 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6663940_3227_4b6c_9569_69678c713609.slice/crio-65c21a7f64c3dcd31a149e5bb10db0ef2effa20a9d75c3d13c95fc816eec09b6 WatchSource:0}: Error finding container 65c21a7f64c3dcd31a149e5bb10db0ef2effa20a9d75c3d13c95fc816eec09b6: Status 404 returned error can't find the container with id 65c21a7f64c3dcd31a149e5bb10db0ef2effa20a9d75c3d13c95fc816eec09b6 Jul 29 00:36:55.553706 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:55.553675 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hzj7z"] Jul 29 00:36:55.557144 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:55.557115 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93783ee9_0737_44f5_a9e3_51438955f859.slice/crio-55c52a03301e2a702916cef36fe789080f5cf6057d2b2901178a2b62526b397f WatchSource:0}: Error finding container 55c52a03301e2a702916cef36fe789080f5cf6057d2b2901178a2b62526b397f: Status 404 returned error can't find the container with id 55c52a03301e2a702916cef36fe789080f5cf6057d2b2901178a2b62526b397f Jul 29 00:36:56.092109 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:56.092016 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hzj7z" event={"ID":"93783ee9-0737-44f5-a9e3-51438955f859","Type":"ContainerStarted","Data":"55c52a03301e2a702916cef36fe789080f5cf6057d2b2901178a2b62526b397f"} Jul 29 00:36:56.092286 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:56.092253 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" event={"ID":"a6663940-3227-4b6c-9569-69678c713609","Type":"ContainerStarted","Data":"65c21a7f64c3dcd31a149e5bb10db0ef2effa20a9d75c3d13c95fc816eec09b6"} Jul 29 00:36:56.095682 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:56.095550 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77879686d6-qmm2n" event={"ID":"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6","Type":"ContainerStarted","Data":"04a0bd99ff26a0220e1269398143af0512b5f1dcf906ba1d68a709fcafbbe009"} Jul 29 00:36:56.095682 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:56.095594 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77879686d6-qmm2n" event={"ID":"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6","Type":"ContainerStarted","Data":"3e0594451135d4814bf7e4052e45242a0bd0c13f16f5dc23371a754aea4250d4"} Jul 29 00:36:56.095682 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:56.095642 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:36:56.116387 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:56.116336 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77879686d6-qmm2n" podStartSLOduration=66.116296226 podStartE2EDuration="1m6.116296226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:35:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:56.115242258 +0000 UTC m=+67.774468148" watchObservedRunningTime="2026-07-29 00:36:56.116296226 +0000 UTC m=+67.775522120" Jul 29 00:36:57.102369 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:57.102329 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mgvr7" event={"ID":"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c","Type":"ContainerStarted","Data":"2d85768be4711a46b9c973f5693c22804f6fbbea1a3f6fc39659298f850bb59d"} Jul 29 00:36:57.102720 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:57.102395 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mgvr7" event={"ID":"a00c6eb4-e6cb-4486-949f-f4f4a7ba538c","Type":"ContainerStarted","Data":"63b41d8ae510595ef3dac0364197832ee17c34c0d4f8ad4f2dcb8add1c25e0fe"} Jul 29 00:36:57.118598 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:57.118547 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mgvr7" podStartSLOduration=66.923490984 podStartE2EDuration="1m8.118526312s" totalImagesPullingTime="1.195035328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:35:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:57.117138999 +0000 UTC m=+68.776364893" watchObservedRunningTime="2026-07-29 00:36:57.118526312 +0000 UTC m=+68.777752206" Jul 29 00:36:58.108227 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.108181 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hzj7z" event={"ID":"93783ee9-0737-44f5-a9e3-51438955f859","Type":"ContainerStarted","Data":"1dd6efa3e8053d8f78c7f101840954e185ab4620a8c71a7f8d239688457b054b"} Jul 29 00:36:58.110047 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.110008 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" event={"ID":"a6663940-3227-4b6c-9569-69678c713609","Type":"ContainerStarted","Data":"a1a4a0278f3aebf3101f7b7dbb2b14b15d15806870c87fc2aba0d6992c4a24e3"} Jul 29 00:36:58.126351 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.126241 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rzc9z" podStartSLOduration=35.646243549 podStartE2EDuration="37.126221196s" totalImagesPullingTime="1.479977647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:58.124731612 +0000 UTC m=+69.783957506" watchObservedRunningTime="2026-07-29 00:36:58.126221196 +0000 UTC m=+69.785447090" Jul 29 00:36:58.514709 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.514605 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z"] Jul 29 00:36:58.518017 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.517987 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z" Jul 29 00:36:58.520409 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.520382 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf"] Jul 29 00:36:58.520636 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.520615 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 29 00:36:58.520752 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.520725 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-bfdgq\"" Jul 29 00:36:58.520951 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.520840 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 29 00:36:58.520951 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.520925 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 29 00:36:58.524800 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.524276 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" Jul 29 00:36:58.527350 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.527299 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm"] Jul 29 00:36:58.529751 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.529728 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 29 00:36:58.530041 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.530020 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.530330 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.530783 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.532770 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv"] Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.532907 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm" Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.535966 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.536003 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xd27k\"" Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.536007 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.536110 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.536008 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z"] Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.538151 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.538205 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.538359 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf"] Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.538546 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.538720 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.539067 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 29 00:36:58.539349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.539256 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 29 00:36:58.546124 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.545399 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm"] Jul 29 00:36:58.551158 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.551133 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv"] Jul 29 00:36:58.555326 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.555288 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh"] Jul 29 00:36:58.559547 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.559525 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh" Jul 29 00:36:58.566035 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.566004 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 29 00:36:58.566140 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.566039 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-g8zld\"" Jul 29 00:36:58.579685 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.579468 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh"] Jul 29 00:36:58.580716 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.580692 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e2bda714-5f55-4541-a6a2-00eec8ef54ee-klusterlet-config\") pod \"klusterlet-addon-workmgr-5778dbc8ff-n96kf\" (UID: \"e2bda714-5f55-4541-a6a2-00eec8ef54ee\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" Jul 29 00:36:58.580821 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.580734 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpfc9\" (UniqueName: \"kubernetes.io/projected/919d1546-a4fa-4afd-8a91-3dfd7ea3c564-kube-api-access-lpfc9\") pod \"cluster-samples-operator-6748489656-vzj4z\" (UID: \"919d1546-a4fa-4afd-8a91-3dfd7ea3c564\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z" Jul 29 00:36:58.580821 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.580770 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/919d1546-a4fa-4afd-8a91-3dfd7ea3c564-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vzj4z\" (UID: \"919d1546-a4fa-4afd-8a91-3dfd7ea3c564\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z" Jul 29 00:36:58.580821 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.580800 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2bda714-5f55-4541-a6a2-00eec8ef54ee-tmp\") pod \"klusterlet-addon-workmgr-5778dbc8ff-n96kf\" (UID: \"e2bda714-5f55-4541-a6a2-00eec8ef54ee\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" Jul 29 00:36:58.580964 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.580879 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv4bg\" (UniqueName: \"kubernetes.io/projected/e2bda714-5f55-4541-a6a2-00eec8ef54ee-kube-api-access-kv4bg\") pod \"klusterlet-addon-workmgr-5778dbc8ff-n96kf\" (UID: \"e2bda714-5f55-4541-a6a2-00eec8ef54ee\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" Jul 29 00:36:58.628022 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.627987 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-t27cm"] Jul 29 00:36:58.631288 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.631259 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-t27cm" Jul 29 00:36:58.632717 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.632691 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-tnffv"] Jul 29 00:36:58.635735 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.635715 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-tnffv" Jul 29 00:36:58.636777 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.636755 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 29 00:36:58.637112 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.637094 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 29 00:36:58.637544 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.637526 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-vcjnz\"" Jul 29 00:36:58.637673 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.637654 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 29 00:36:58.637755 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.637670 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-n56fp\"" Jul 29 00:36:58.638024 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.638001 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 29 00:36:58.652851 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.652830 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-lsjv6"] Jul 29 00:36:58.657072 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.657052 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5479f5c76c-7vtwf"] Jul 29 00:36:58.657180 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.657154 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.660382 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.660363 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-tnffv"] Jul 29 00:36:58.660477 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.660398 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-t27cm"] Jul 29 00:36:58.660534 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.660477 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.666329 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.666288 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 29 00:36:58.666685 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.666666 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 29 00:36:58.666780 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.666760 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 29 00:36:58.666902 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.666884 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 29 00:36:58.666980 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.666955 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 29 00:36:58.667940 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.667829 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 29 00:36:58.667940 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.667869 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 29 00:36:58.668614 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.668592 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 29 00:36:58.668820 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.668797 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 29 00:36:58.668820 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.668802 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-g9vnl\"" Jul 29 00:36:58.672517 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.672502 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 29 00:36:58.672644 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.672551 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ltprj\"" Jul 29 00:36:58.676082 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.676029 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-lsjv6"] Jul 29 00:36:58.677793 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.677725 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 29 00:36:58.681247 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.681222 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3e33561-3dc0-4538-acba-55fcb83b3c04-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-xdvdm\" (UID: \"f3e33561-3dc0-4538-acba-55fcb83b3c04\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm" Jul 29 00:36:58.681351 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.681276 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv4bg\" (UniqueName: \"kubernetes.io/projected/e2bda714-5f55-4541-a6a2-00eec8ef54ee-kube-api-access-kv4bg\") pod \"klusterlet-addon-workmgr-5778dbc8ff-n96kf\" (UID: \"e2bda714-5f55-4541-a6a2-00eec8ef54ee\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" Jul 29 00:36:58.681351 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.681344 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/35aa65bc-596b-4092-8908-faebb404cd0a-hub\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.681471 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.681424 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/35aa65bc-596b-4092-8908-faebb404cd0a-ca\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.681471 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.681461 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35aa65bc-596b-4092-8908-faebb404cd0a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.681584 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.681499 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcmdv\" (UniqueName: \"kubernetes.io/projected/fcf4232a-0bc5-4958-89d9-1f7e8a0be177-kube-api-access-gcmdv\") pod \"managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh\" (UID: \"fcf4232a-0bc5-4958-89d9-1f7e8a0be177\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh" Jul 29 00:36:58.681584 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.681555 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e2bda714-5f55-4541-a6a2-00eec8ef54ee-klusterlet-config\") pod \"klusterlet-addon-workmgr-5778dbc8ff-n96kf\" (UID: \"e2bda714-5f55-4541-a6a2-00eec8ef54ee\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" Jul 29 00:36:58.681687 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.681586 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpfc9\" (UniqueName: \"kubernetes.io/projected/919d1546-a4fa-4afd-8a91-3dfd7ea3c564-kube-api-access-lpfc9\") pod \"cluster-samples-operator-6748489656-vzj4z\" (UID: \"919d1546-a4fa-4afd-8a91-3dfd7ea3c564\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z" Jul 29 00:36:58.681687 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.681623 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/919d1546-a4fa-4afd-8a91-3dfd7ea3c564-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vzj4z\" (UID: \"919d1546-a4fa-4afd-8a91-3dfd7ea3c564\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z" Jul 29 00:36:58.681912 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.681844 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fcf4232a-0bc5-4958-89d9-1f7e8a0be177-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh\" (UID: \"fcf4232a-0bc5-4958-89d9-1f7e8a0be177\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh" Jul 29 00:36:58.681912 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.681894 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/35aa65bc-596b-4092-8908-faebb404cd0a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.682077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.681935 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pjln\" (UniqueName: \"kubernetes.io/projected/514294bb-9bd1-4b6a-975f-f65446febeb1-kube-api-access-9pjln\") pod \"volume-data-source-validator-549c6d54cf-t27cm\" (UID: \"514294bb-9bd1-4b6a-975f-f65446febeb1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-t27cm" Jul 29 00:36:58.682077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.681979 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-895jb\" (UniqueName: \"kubernetes.io/projected/f3e33561-3dc0-4538-acba-55fcb83b3c04-kube-api-access-895jb\") pod \"cluster-monitoring-operator-548b489cb8-xdvdm\" (UID: \"f3e33561-3dc0-4538-acba-55fcb83b3c04\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm" Jul 29 00:36:58.682077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.682007 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2bda714-5f55-4541-a6a2-00eec8ef54ee-tmp\") pod \"klusterlet-addon-workmgr-5778dbc8ff-n96kf\" (UID: \"e2bda714-5f55-4541-a6a2-00eec8ef54ee\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" Jul 29 00:36:58.682077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.682063 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/35aa65bc-596b-4092-8908-faebb404cd0a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.682261 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.682146 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f3e33561-3dc0-4538-acba-55fcb83b3c04-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-xdvdm\" (UID: \"f3e33561-3dc0-4538-acba-55fcb83b3c04\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm" Jul 29 00:36:58.682261 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.682179 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22j2d\" (UniqueName: \"kubernetes.io/projected/35aa65bc-596b-4092-8908-faebb404cd0a-kube-api-access-22j2d\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.682436 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.682394 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2bda714-5f55-4541-a6a2-00eec8ef54ee-tmp\") pod \"klusterlet-addon-workmgr-5778dbc8ff-n96kf\" (UID: \"e2bda714-5f55-4541-a6a2-00eec8ef54ee\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" Jul 29 00:36:58.684507 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.684488 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/919d1546-a4fa-4afd-8a91-3dfd7ea3c564-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vzj4z\" (UID: \"919d1546-a4fa-4afd-8a91-3dfd7ea3c564\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z" Jul 29 00:36:58.684624 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.684601 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e2bda714-5f55-4541-a6a2-00eec8ef54ee-klusterlet-config\") pod \"klusterlet-addon-workmgr-5778dbc8ff-n96kf\" (UID: \"e2bda714-5f55-4541-a6a2-00eec8ef54ee\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" Jul 29 00:36:58.695373 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.695347 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5479f5c76c-7vtwf"] Jul 29 00:36:58.701337 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.701236 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpfc9\" (UniqueName: \"kubernetes.io/projected/919d1546-a4fa-4afd-8a91-3dfd7ea3c564-kube-api-access-lpfc9\") pod \"cluster-samples-operator-6748489656-vzj4z\" (UID: \"919d1546-a4fa-4afd-8a91-3dfd7ea3c564\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z" Jul 29 00:36:58.705199 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.705180 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv4bg\" (UniqueName: \"kubernetes.io/projected/e2bda714-5f55-4541-a6a2-00eec8ef54ee-kube-api-access-kv4bg\") pod \"klusterlet-addon-workmgr-5778dbc8ff-n96kf\" (UID: \"e2bda714-5f55-4541-a6a2-00eec8ef54ee\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" Jul 29 00:36:58.782763 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.782682 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fcf4232a-0bc5-4958-89d9-1f7e8a0be177-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh\" (UID: \"fcf4232a-0bc5-4958-89d9-1f7e8a0be177\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh" Jul 29 00:36:58.782763 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.782732 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/35aa65bc-596b-4092-8908-faebb404cd0a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.782763 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.782760 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9da12546-7d93-4a97-a05f-0a7c83e3f782-serving-cert\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.783055 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.782783 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/34df441f-d815-4b61-b519-02120ab837ea-default-certificate\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.783055 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.782810 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pjln\" (UniqueName: \"kubernetes.io/projected/514294bb-9bd1-4b6a-975f-f65446febeb1-kube-api-access-9pjln\") pod \"volume-data-source-validator-549c6d54cf-t27cm\" (UID: \"514294bb-9bd1-4b6a-975f-f65446febeb1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-t27cm" Jul 29 00:36:58.783055 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.782833 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-895jb\" (UniqueName: \"kubernetes.io/projected/f3e33561-3dc0-4538-acba-55fcb83b3c04-kube-api-access-895jb\") pod \"cluster-monitoring-operator-548b489cb8-xdvdm\" (UID: \"f3e33561-3dc0-4538-acba-55fcb83b3c04\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm" Jul 29 00:36:58.783055 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.782855 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8996t\" (UniqueName: \"kubernetes.io/projected/34df441f-d815-4b61-b519-02120ab837ea-kube-api-access-8996t\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.783055 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.782882 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shzl7\" (UniqueName: \"kubernetes.io/projected/9da12546-7d93-4a97-a05f-0a7c83e3f782-kube-api-access-shzl7\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.783055 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.782905 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/35aa65bc-596b-4092-8908-faebb404cd0a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.783055 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.782926 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9da12546-7d93-4a97-a05f-0a7c83e3f782-service-ca-bundle\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.783055 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.782948 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34df441f-d815-4b61-b519-02120ab837ea-service-ca-bundle\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.783055 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.782985 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f3e33561-3dc0-4538-acba-55fcb83b3c04-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-xdvdm\" (UID: \"f3e33561-3dc0-4538-acba-55fcb83b3c04\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm" Jul 29 00:36:58.783055 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.783015 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22j2d\" (UniqueName: \"kubernetes.io/projected/35aa65bc-596b-4092-8908-faebb404cd0a-kube-api-access-22j2d\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.783055 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.783050 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3e33561-3dc0-4538-acba-55fcb83b3c04-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-xdvdm\" (UID: \"f3e33561-3dc0-4538-acba-55fcb83b3c04\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm" Jul 29 00:36:58.783599 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.783090 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/35aa65bc-596b-4092-8908-faebb404cd0a-hub\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.783599 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.783146 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bvkl\" (UniqueName: \"kubernetes.io/projected/0e291f3f-222d-49da-8d49-f6ca675b45c2-kube-api-access-6bvkl\") pod \"downloads-7c97dc4555-tnffv\" (UID: \"0e291f3f-222d-49da-8d49-f6ca675b45c2\") " pod="openshift-console/downloads-7c97dc4555-tnffv" Jul 29 00:36:58.783599 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.783175 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9da12546-7d93-4a97-a05f-0a7c83e3f782-snapshots\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.783599 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.783199 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/35aa65bc-596b-4092-8908-faebb404cd0a-ca\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.783599 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.783214 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35aa65bc-596b-4092-8908-faebb404cd0a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.783599 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.783229 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9da12546-7d93-4a97-a05f-0a7c83e3f782-tmp\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.783599 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.783247 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/34df441f-d815-4b61-b519-02120ab837ea-stats-auth\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.783599 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.783266 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcmdv\" (UniqueName: \"kubernetes.io/projected/fcf4232a-0bc5-4958-89d9-1f7e8a0be177-kube-api-access-gcmdv\") pod \"managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh\" (UID: \"fcf4232a-0bc5-4958-89d9-1f7e8a0be177\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh" Jul 29 00:36:58.783599 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.783287 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34df441f-d815-4b61-b519-02120ab837ea-metrics-certs\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.783599 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.783345 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9da12546-7d93-4a97-a05f-0a7c83e3f782-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.783599 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.783537 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/35aa65bc-596b-4092-8908-faebb404cd0a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.784326 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.784285 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f3e33561-3dc0-4538-acba-55fcb83b3c04-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-xdvdm\" (UID: \"f3e33561-3dc0-4538-acba-55fcb83b3c04\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm" Jul 29 00:36:58.786007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.785972 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/35aa65bc-596b-4092-8908-faebb404cd0a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.786233 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.786209 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/35aa65bc-596b-4092-8908-faebb404cd0a-ca\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.786352 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.786209 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fcf4232a-0bc5-4958-89d9-1f7e8a0be177-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh\" (UID: \"fcf4232a-0bc5-4958-89d9-1f7e8a0be177\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh" Jul 29 00:36:58.787132 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.787057 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35aa65bc-596b-4092-8908-faebb404cd0a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.787232 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.787121 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/35aa65bc-596b-4092-8908-faebb404cd0a-hub\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.787454 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.787429 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3e33561-3dc0-4538-acba-55fcb83b3c04-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-xdvdm\" (UID: \"f3e33561-3dc0-4538-acba-55fcb83b3c04\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm" Jul 29 00:36:58.795908 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.795885 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-895jb\" (UniqueName: \"kubernetes.io/projected/f3e33561-3dc0-4538-acba-55fcb83b3c04-kube-api-access-895jb\") pod \"cluster-monitoring-operator-548b489cb8-xdvdm\" (UID: \"f3e33561-3dc0-4538-acba-55fcb83b3c04\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm" Jul 29 00:36:58.796215 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.796185 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22j2d\" (UniqueName: \"kubernetes.io/projected/35aa65bc-596b-4092-8908-faebb404cd0a-kube-api-access-22j2d\") pod \"cluster-proxy-proxy-agent-848dcf56cb-p58rv\" (UID: \"35aa65bc-596b-4092-8908-faebb404cd0a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.796422 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.796393 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pjln\" (UniqueName: \"kubernetes.io/projected/514294bb-9bd1-4b6a-975f-f65446febeb1-kube-api-access-9pjln\") pod \"volume-data-source-validator-549c6d54cf-t27cm\" (UID: \"514294bb-9bd1-4b6a-975f-f65446febeb1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-t27cm" Jul 29 00:36:58.797073 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.797053 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcmdv\" (UniqueName: \"kubernetes.io/projected/fcf4232a-0bc5-4958-89d9-1f7e8a0be177-kube-api-access-gcmdv\") pod \"managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh\" (UID: \"fcf4232a-0bc5-4958-89d9-1f7e8a0be177\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh" Jul 29 00:36:58.835885 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.835850 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z" Jul 29 00:36:58.850504 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.850474 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" Jul 29 00:36:58.859619 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.859596 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm" Jul 29 00:36:58.874278 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.874258 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" Jul 29 00:36:58.881590 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.881573 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh" Jul 29 00:36:58.883710 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.883687 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9da12546-7d93-4a97-a05f-0a7c83e3f782-snapshots\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.883831 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.883726 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9da12546-7d93-4a97-a05f-0a7c83e3f782-tmp\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.883831 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.883787 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/34df441f-d815-4b61-b519-02120ab837ea-stats-auth\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.883831 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.883806 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34df441f-d815-4b61-b519-02120ab837ea-metrics-certs\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.883831 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.883828 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9da12546-7d93-4a97-a05f-0a7c83e3f782-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.884039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.883863 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9da12546-7d93-4a97-a05f-0a7c83e3f782-serving-cert\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.884039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.883886 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/34df441f-d815-4b61-b519-02120ab837ea-default-certificate\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.884039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.883912 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8996t\" (UniqueName: \"kubernetes.io/projected/34df441f-d815-4b61-b519-02120ab837ea-kube-api-access-8996t\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.884039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.883940 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shzl7\" (UniqueName: \"kubernetes.io/projected/9da12546-7d93-4a97-a05f-0a7c83e3f782-kube-api-access-shzl7\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.884039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.883964 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9da12546-7d93-4a97-a05f-0a7c83e3f782-service-ca-bundle\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.884039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.883992 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34df441f-d815-4b61-b519-02120ab837ea-service-ca-bundle\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.884331 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.884066 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bvkl\" (UniqueName: \"kubernetes.io/projected/0e291f3f-222d-49da-8d49-f6ca675b45c2-kube-api-access-6bvkl\") pod \"downloads-7c97dc4555-tnffv\" (UID: \"0e291f3f-222d-49da-8d49-f6ca675b45c2\") " pod="openshift-console/downloads-7c97dc4555-tnffv" Jul 29 00:36:58.884488 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.884455 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9da12546-7d93-4a97-a05f-0a7c83e3f782-snapshots\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.885028 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.884970 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9da12546-7d93-4a97-a05f-0a7c83e3f782-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.885103 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.885062 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9da12546-7d93-4a97-a05f-0a7c83e3f782-service-ca-bundle\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.885349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.885325 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34df441f-d815-4b61-b519-02120ab837ea-service-ca-bundle\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.885444 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.885359 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9da12546-7d93-4a97-a05f-0a7c83e3f782-tmp\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.887486 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.887454 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/34df441f-d815-4b61-b519-02120ab837ea-default-certificate\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.887582 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.887455 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/34df441f-d815-4b61-b519-02120ab837ea-stats-auth\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.887582 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.887455 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34df441f-d815-4b61-b519-02120ab837ea-metrics-certs\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.887722 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.887688 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9da12546-7d93-4a97-a05f-0a7c83e3f782-serving-cert\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.895406 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.895383 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shzl7\" (UniqueName: \"kubernetes.io/projected/9da12546-7d93-4a97-a05f-0a7c83e3f782-kube-api-access-shzl7\") pod \"insights-operator-5885fdb759-lsjv6\" (UID: \"9da12546-7d93-4a97-a05f-0a7c83e3f782\") " pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.896096 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.896072 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bvkl\" (UniqueName: \"kubernetes.io/projected/0e291f3f-222d-49da-8d49-f6ca675b45c2-kube-api-access-6bvkl\") pod \"downloads-7c97dc4555-tnffv\" (UID: \"0e291f3f-222d-49da-8d49-f6ca675b45c2\") " pod="openshift-console/downloads-7c97dc4555-tnffv" Jul 29 00:36:58.897794 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.897768 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8996t\" (UniqueName: \"kubernetes.io/projected/34df441f-d815-4b61-b519-02120ab837ea-kube-api-access-8996t\") pod \"router-default-5479f5c76c-7vtwf\" (UID: \"34df441f-d815-4b61-b519-02120ab837ea\") " pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:58.958386 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.958341 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-t27cm" Jul 29 00:36:58.965694 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.965663 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-tnffv" Jul 29 00:36:58.975912 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.975885 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-lsjv6" Jul 29 00:36:58.982764 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:58.982732 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:36:59.628587 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:59.628528 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh"] Jul 29 00:36:59.797236 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:59.797209 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm"] Jul 29 00:36:59.803003 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:59.802952 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3e33561_3dc0_4538_acba_55fcb83b3c04.slice/crio-ce031241e3dc4830abda3a9eb0e39090ddb60b305037926a23da4d00f290920a WatchSource:0}: Error finding container ce031241e3dc4830abda3a9eb0e39090ddb60b305037926a23da4d00f290920a: Status 404 returned error can't find the container with id ce031241e3dc4830abda3a9eb0e39090ddb60b305037926a23da4d00f290920a Jul 29 00:36:59.829851 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:59.829723 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z"] Jul 29 00:36:59.987772 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:59.987702 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-lsjv6"] Jul 29 00:36:59.991442 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:59.991407 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9da12546_7d93_4a97_a05f_0a7c83e3f782.slice/crio-dd07127343e8479fe38916359b9d86907c9aa058cef2646d96f374f29792499f WatchSource:0}: Error finding container dd07127343e8479fe38916359b9d86907c9aa058cef2646d96f374f29792499f: Status 404 returned error can't find the container with id dd07127343e8479fe38916359b9d86907c9aa058cef2646d96f374f29792499f Jul 29 00:36:59.992006 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:36:59.991958 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5479f5c76c-7vtwf"] Jul 29 00:36:59.993856 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:36:59.993820 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34df441f_d815_4b61_b519_02120ab837ea.slice/crio-d8d4db2679d205c45c98611fa8165dc3612db43bf7e8ca0cb5c34926f5569690 WatchSource:0}: Error finding container d8d4db2679d205c45c98611fa8165dc3612db43bf7e8ca0cb5c34926f5569690: Status 404 returned error can't find the container with id d8d4db2679d205c45c98611fa8165dc3612db43bf7e8ca0cb5c34926f5569690 Jul 29 00:37:00.058122 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.058092 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf"] Jul 29 00:37:00.062718 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:37:00.062686 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2bda714_5f55_4541_a6a2_00eec8ef54ee.slice/crio-be1dcf98b0585253d260cf34bc18da687a841638a47cdef9316abef55f5101f3 WatchSource:0}: Error finding container be1dcf98b0585253d260cf34bc18da687a841638a47cdef9316abef55f5101f3: Status 404 returned error can't find the container with id be1dcf98b0585253d260cf34bc18da687a841638a47cdef9316abef55f5101f3 Jul 29 00:37:00.065299 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.065243 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-t27cm"] Jul 29 00:37:00.069086 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.068936 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv"] Jul 29 00:37:00.070342 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:37:00.070288 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod514294bb_9bd1_4b6a_975f_f65446febeb1.slice/crio-b854c8fc8ed2b13b33b1d11f06bc61a91dd01bd0a943c46ef37f031dbdce8b9e WatchSource:0}: Error finding container b854c8fc8ed2b13b33b1d11f06bc61a91dd01bd0a943c46ef37f031dbdce8b9e: Status 404 returned error can't find the container with id b854c8fc8ed2b13b33b1d11f06bc61a91dd01bd0a943c46ef37f031dbdce8b9e Jul 29 00:37:00.072022 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:37:00.071984 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35aa65bc_596b_4092_8908_faebb404cd0a.slice/crio-ac497478045f1ba20c597dcb6dca41dbc53b7b9ac90dab2965ddcb17e866a7ea WatchSource:0}: Error finding container ac497478045f1ba20c597dcb6dca41dbc53b7b9ac90dab2965ddcb17e866a7ea: Status 404 returned error can't find the container with id ac497478045f1ba20c597dcb6dca41dbc53b7b9ac90dab2965ddcb17e866a7ea Jul 29 00:37:00.077216 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.077192 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-tnffv"] Jul 29 00:37:00.081917 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:37:00.081859 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e291f3f_222d_49da_8d49_f6ca675b45c2.slice/crio-5addcccdd684dc7b28e7a2b1e8ef7b72c2ced76f0025db3f43922e695fcae733 WatchSource:0}: Error finding container 5addcccdd684dc7b28e7a2b1e8ef7b72c2ced76f0025db3f43922e695fcae733: Status 404 returned error can't find the container with id 5addcccdd684dc7b28e7a2b1e8ef7b72c2ced76f0025db3f43922e695fcae733 Jul 29 00:37:00.120212 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.120165 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm" event={"ID":"f3e33561-3dc0-4538-acba-55fcb83b3c04","Type":"ContainerStarted","Data":"ce031241e3dc4830abda3a9eb0e39090ddb60b305037926a23da4d00f290920a"} Jul 29 00:37:00.122730 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.122693 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-tnffv" event={"ID":"0e291f3f-222d-49da-8d49-f6ca675b45c2","Type":"ContainerStarted","Data":"5addcccdd684dc7b28e7a2b1e8ef7b72c2ced76f0025db3f43922e695fcae733"} Jul 29 00:37:00.124275 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.124250 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5479f5c76c-7vtwf" event={"ID":"34df441f-d815-4b61-b519-02120ab837ea","Type":"ContainerStarted","Data":"a242ad6140a16000be44eaf266863d5a13a0204a0618a6ca6151885aef5d35b0"} Jul 29 00:37:00.124386 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.124287 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5479f5c76c-7vtwf" event={"ID":"34df441f-d815-4b61-b519-02120ab837ea","Type":"ContainerStarted","Data":"d8d4db2679d205c45c98611fa8165dc3612db43bf7e8ca0cb5c34926f5569690"} Jul 29 00:37:00.125842 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.125810 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jghtb" event={"ID":"a50c1131-277e-4dd2-9fdd-dfd5d5aabdee","Type":"ContainerStarted","Data":"c1a8097951b6cf4538bae9a54a1ad094b63b5bed6b01ca5070d66ba9dc82e4f0"} Jul 29 00:37:00.127116 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.127078 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh" event={"ID":"fcf4232a-0bc5-4958-89d9-1f7e8a0be177","Type":"ContainerStarted","Data":"8ac1001db6b1fe77b637cd42b05dbf2fbdba1aa1f98f87a6f0f77c451d9b6ae6"} Jul 29 00:37:00.128187 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.128166 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" event={"ID":"35aa65bc-596b-4092-8908-faebb404cd0a","Type":"ContainerStarted","Data":"ac497478045f1ba20c597dcb6dca41dbc53b7b9ac90dab2965ddcb17e866a7ea"} Jul 29 00:37:00.129229 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.129211 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-t27cm" event={"ID":"514294bb-9bd1-4b6a-975f-f65446febeb1","Type":"ContainerStarted","Data":"b854c8fc8ed2b13b33b1d11f06bc61a91dd01bd0a943c46ef37f031dbdce8b9e"} Jul 29 00:37:00.130244 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.130225 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" event={"ID":"e2bda714-5f55-4541-a6a2-00eec8ef54ee","Type":"ContainerStarted","Data":"be1dcf98b0585253d260cf34bc18da687a841638a47cdef9316abef55f5101f3"} Jul 29 00:37:00.131844 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.131822 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hzj7z" event={"ID":"93783ee9-0737-44f5-a9e3-51438955f859","Type":"ContainerStarted","Data":"f3cae0ae384aaffeddfcf8c989decb7c54d7bdca4721f631dae38c38f89aa9a9"} Jul 29 00:37:00.131978 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.131962 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hzj7z" Jul 29 00:37:00.133043 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.133019 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z" event={"ID":"919d1546-a4fa-4afd-8a91-3dfd7ea3c564","Type":"ContainerStarted","Data":"595e2a1a0bce51ea3c70357eef657c4d27a35e24d20aab9c7cf83a16a4139072"} Jul 29 00:37:00.134109 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.134084 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-lsjv6" event={"ID":"9da12546-7d93-4a97-a05f-0a7c83e3f782","Type":"ContainerStarted","Data":"dd07127343e8479fe38916359b9d86907c9aa058cef2646d96f374f29792499f"} Jul 29 00:37:00.146222 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.146179 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5479f5c76c-7vtwf" podStartSLOduration=2.146165603 podStartE2EDuration="2.146165603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:00.146119303 +0000 UTC m=+71.805345197" watchObservedRunningTime="2026-07-29 00:37:00.146165603 +0000 UTC m=+71.805391496" Jul 29 00:37:00.168736 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.168690 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hzj7z" podStartSLOduration=34.863478912 podStartE2EDuration="37.168673892s" totalImagesPullingTime="2.30519498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:00.168506898 +0000 UTC m=+71.827732793" watchObservedRunningTime="2026-07-29 00:37:00.168673892 +0000 UTC m=+71.827899787" Jul 29 00:37:00.186175 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.186130 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jghtb" podStartSLOduration=66.642143235 podStartE2EDuration="1m11.186117807s" totalImagesPullingTime="4.543974572s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:35:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:00.18538048 +0000 UTC m=+71.844606373" watchObservedRunningTime="2026-07-29 00:37:00.186117807 +0000 UTC m=+71.845343703" Jul 29 00:37:00.984103 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.984007 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:37:00.987103 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:00.987076 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:37:01.152329 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:01.152237 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:37:01.155502 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:01.155293 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5479f5c76c-7vtwf" Jul 29 00:37:07.154842 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:07.154809 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hzj7z" Jul 29 00:37:09.217475 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.217273 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z" event={"ID":"919d1546-a4fa-4afd-8a91-3dfd7ea3c564","Type":"ContainerStarted","Data":"a47b890d6da4f185d02900a28dbacff1d0aa14efb19b195c1fee71fb5e796a7d"} Jul 29 00:37:09.224115 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.224083 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-lsjv6" event={"ID":"9da12546-7d93-4a97-a05f-0a7c83e3f782","Type":"ContainerStarted","Data":"7188ad06284eeedcbf43bd482925bf762f3b47275fb2bc457fc59b067b2633f5"} Jul 29 00:37:09.232696 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.232667 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm" event={"ID":"f3e33561-3dc0-4538-acba-55fcb83b3c04","Type":"ContainerStarted","Data":"dd85d95ee9b5dd413ca0cfbc4b2771fcb5f62af741cae953952130a37cf9111f"} Jul 29 00:37:09.237691 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.237658 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" event={"ID":"35aa65bc-596b-4092-8908-faebb404cd0a","Type":"ContainerStarted","Data":"fba4bce716e21fd7f88ee29c451b1ffb3e4211af9dac94b972e7331bf180300f"} Jul 29 00:37:09.241350 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.240540 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-t27cm" event={"ID":"514294bb-9bd1-4b6a-975f-f65446febeb1","Type":"ContainerStarted","Data":"a1462de09ff1344213b1da3d28ca53a977a9100f175d55ae4e91e41e14c741cc"} Jul 29 00:37:09.243187 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.243163 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" event={"ID":"e2bda714-5f55-4541-a6a2-00eec8ef54ee","Type":"ContainerStarted","Data":"9d2c48e16903ced29ccb76c25b2d8d482179f67afc144cb5328370a80f6b4a35"} Jul 29 00:37:09.247609 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.246604 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" Jul 29 00:37:09.247609 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.245324 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-lsjv6" podStartSLOduration=2.248340284 podStartE2EDuration="11.245289631s" totalImagesPullingTime="8.996949347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:09.243649266 +0000 UTC m=+80.902875160" watchObservedRunningTime="2026-07-29 00:37:09.245289631 +0000 UTC m=+80.904515524" Jul 29 00:37:09.249962 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.249942 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" Jul 29 00:37:09.262835 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.262739 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-xdvdm" podStartSLOduration=2.077939872 podStartE2EDuration="11.262717594s" totalImagesPullingTime="9.184777722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:09.260081881 +0000 UTC m=+80.919307775" watchObservedRunningTime="2026-07-29 00:37:09.262717594 +0000 UTC m=+80.921943479" Jul 29 00:37:09.283010 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.282047 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-t27cm" podStartSLOduration=2.365114942 podStartE2EDuration="11.282028001s" totalImagesPullingTime="8.916913059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:09.281478261 +0000 UTC m=+80.940704155" watchObservedRunningTime="2026-07-29 00:37:09.282028001 +0000 UTC m=+80.941253893" Jul 29 00:37:09.732722 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.732398 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5778dbc8ff-n96kf" podStartSLOduration=2.778381026 podStartE2EDuration="11.73237528s" totalImagesPullingTime="8.953994254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:09.297629026 +0000 UTC m=+80.956854919" watchObservedRunningTime="2026-07-29 00:37:09.73237528 +0000 UTC m=+81.391601175" Jul 29 00:37:09.733404 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.733290 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wtq82"] Jul 29 00:37:09.737365 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.737294 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:09.740210 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.740028 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 29 00:37:09.740210 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.740028 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-krv7p\"" Jul 29 00:37:09.742696 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.742531 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 29 00:37:09.751187 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.751080 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wtq82"] Jul 29 00:37:09.907284 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.907242 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a095e234-21c3-4383-ad01-d399215129a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:09.907486 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.907291 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4s9q\" (UniqueName: \"kubernetes.io/projected/a095e234-21c3-4383-ad01-d399215129a0-kube-api-access-c4s9q\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:09.907486 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.907341 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a095e234-21c3-4383-ad01-d399215129a0-data-volume\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:09.907486 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.907365 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a095e234-21c3-4383-ad01-d399215129a0-crio-socket\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:09.907486 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:09.907392 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a095e234-21c3-4383-ad01-d399215129a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:10.008199 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.008009 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a095e234-21c3-4383-ad01-d399215129a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:10.008199 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.008073 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4s9q\" (UniqueName: \"kubernetes.io/projected/a095e234-21c3-4383-ad01-d399215129a0-kube-api-access-c4s9q\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:10.008199 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.008123 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a095e234-21c3-4383-ad01-d399215129a0-data-volume\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:10.008643 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.008364 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a095e234-21c3-4383-ad01-d399215129a0-crio-socket\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:10.008643 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.008420 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a095e234-21c3-4383-ad01-d399215129a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:10.008643 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.008504 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a095e234-21c3-4383-ad01-d399215129a0-crio-socket\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:10.008643 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:37:10.008574 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 00:37:10.008643 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.008584 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a095e234-21c3-4383-ad01-d399215129a0-data-volume\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:10.008643 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:37:10.008620 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a095e234-21c3-4383-ad01-d399215129a0-insights-runtime-extractor-tls podName:a095e234-21c3-4383-ad01-d399215129a0 nodeName:}" failed. No retries permitted until 2026-07-29 00:37:10.50860381 +0000 UTC m=+82.167829685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a095e234-21c3-4383-ad01-d399215129a0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wtq82" (UID: "a095e234-21c3-4383-ad01-d399215129a0") : secret "insights-runtime-extractor-tls" not found Jul 29 00:37:10.009007 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.008988 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a095e234-21c3-4383-ad01-d399215129a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:10.020765 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.020735 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4s9q\" (UniqueName: \"kubernetes.io/projected/a095e234-21c3-4383-ad01-d399215129a0-kube-api-access-c4s9q\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:10.251209 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.251172 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z" event={"ID":"919d1546-a4fa-4afd-8a91-3dfd7ea3c564","Type":"ContainerStarted","Data":"960978e3631d46ef238af8d8b56f62a369ca8045fe28aca2c30919a578ccdc37"} Jul 29 00:37:10.254845 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.254814 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" event={"ID":"35aa65bc-596b-4092-8908-faebb404cd0a","Type":"ContainerStarted","Data":"ac8c2b29dc98de2824e096ac0c012ae2f4a083686128aff7062ebedc87eda79f"} Jul 29 00:37:10.254965 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.254856 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" event={"ID":"35aa65bc-596b-4092-8908-faebb404cd0a","Type":"ContainerStarted","Data":"020a3930c657f0e2f1651872f6a0490af855994a4671f40abf82f78f17419875"} Jul 29 00:37:10.273005 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.272918 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vzj4z" podStartSLOduration=3.161741444 podStartE2EDuration="12.272898848s" totalImagesPullingTime="9.111157404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:10.271409625 +0000 UTC m=+81.930635516" watchObservedRunningTime="2026-07-29 00:37:10.272898848 +0000 UTC m=+81.932124743" Jul 29 00:37:10.294427 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.293959 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848dcf56cb-p58rv" podStartSLOduration=3.345098304 podStartE2EDuration="12.293939206s" totalImagesPullingTime="8.948840902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:10.292640694 +0000 UTC m=+81.951866589" watchObservedRunningTime="2026-07-29 00:37:10.293939206 +0000 UTC m=+81.953165101" Jul 29 00:37:10.513050 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.513013 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a095e234-21c3-4383-ad01-d399215129a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:10.515779 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.515744 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a095e234-21c3-4383-ad01-d399215129a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wtq82\" (UID: \"a095e234-21c3-4383-ad01-d399215129a0\") " pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:10.655956 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.655922 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wtq82" Jul 29 00:37:10.807080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:10.807053 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wtq82"] Jul 29 00:37:10.809991 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:37:10.809962 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda095e234_21c3_4383_ad01_d399215129a0.slice/crio-3c31c1cc0594518192caff460043a8be43711164d7ec6e8552d138f610902c9d WatchSource:0}: Error finding container 3c31c1cc0594518192caff460043a8be43711164d7ec6e8552d138f610902c9d: Status 404 returned error can't find the container with id 3c31c1cc0594518192caff460043a8be43711164d7ec6e8552d138f610902c9d Jul 29 00:37:11.263242 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:11.263201 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wtq82" event={"ID":"a095e234-21c3-4383-ad01-d399215129a0","Type":"ContainerStarted","Data":"3c31c1cc0594518192caff460043a8be43711164d7ec6e8552d138f610902c9d"} Jul 29 00:37:11.604800 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:11.604765 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-rzc9z_a6663940-3227-4b6c-9569-69678c713609/networking-console-plugin/0.log" Jul 29 00:37:11.803049 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:11.803017 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 00:37:12.011468 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:12.011362 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/2.log" Jul 29 00:37:13.203283 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:13.203247 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hzj7z_93783ee9-0737-44f5-a9e3-51438955f859/dns/0.log" Jul 29 00:37:13.277955 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:13.277916 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh" event={"ID":"fcf4232a-0bc5-4958-89d9-1f7e8a0be177","Type":"ContainerStarted","Data":"dbb3c9110f62cee4f47bc30f4166efcec5f18bf3af19964752371bcac45bfdaf"} Jul 29 00:37:13.293133 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:13.293074 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b4dbfd4-vt6qh" podStartSLOduration=2.7148395709999997 podStartE2EDuration="15.293059836s" totalImagesPullingTime="12.578220265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:13.292127946 +0000 UTC m=+84.951353840" watchObservedRunningTime="2026-07-29 00:37:13.293059836 +0000 UTC m=+84.952285729" Jul 29 00:37:13.403402 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:13.403367 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hzj7z_93783ee9-0737-44f5-a9e3-51438955f859/kube-rbac-proxy/0.log" Jul 29 00:37:14.402854 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:14.402828 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vpqgd_20137ae0-172b-4165-9f8f-87bb7df1fd80/dns-node-resolver/0.log" Jul 29 00:37:14.804001 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:14.803929 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-77879686d6-qmm2n_2fa28369-78d4-43b6-bba0-1ea67ef2a6a6/registry/0.log" Jul 29 00:37:15.333001 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:15.332950 2558 patch_prober.go:28] interesting pod/image-registry-77879686d6-qmm2n container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 29 00:37:15.333171 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:15.333030 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-77879686d6-qmm2n" podUID="2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 00:37:15.403635 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:15.403598 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dffdw_b5888d04-a838-4a13-ac06-5ef697b5ee59/node-ca/0.log" Jul 29 00:37:15.603351 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:15.603262 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5479f5c76c-7vtwf_34df441f-d815-4b61-b519-02120ab837ea/router/0.log" Jul 29 00:37:16.004331 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:16.004189 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6g74f_1939d4b2-0a14-4f02-aa47-98e04683a8dd/serve-healthcheck-canary/0.log" Jul 29 00:37:18.115916 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:18.115886 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:37:24.329032 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:24.328972 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wtq82" event={"ID":"a095e234-21c3-4383-ad01-d399215129a0","Type":"ContainerStarted","Data":"5c146ca8c56655fabcb02d20a723ed5b0cd876464eb6aa5c509bfd998c608f2b"} Jul 29 00:37:24.330905 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:24.330873 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-tnffv" event={"ID":"0e291f3f-222d-49da-8d49-f6ca675b45c2","Type":"ContainerStarted","Data":"39e7c9d21a8e2c7f986e88398aa469301d26984297b4b6e3145e2543abde2c0d"} Jul 29 00:37:24.331121 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:24.331101 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-tnffv" Jul 29 00:37:24.351062 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:24.350999 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-tnffv" podStartSLOduration=2.822276082 podStartE2EDuration="26.350981736s" totalImagesPullingTime="23.528705654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:24.349303698 +0000 UTC m=+96.008529594" watchObservedRunningTime="2026-07-29 00:37:24.350981736 +0000 UTC m=+96.010207630" Jul 29 00:37:24.351618 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:24.351589 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-tnffv" Jul 29 00:37:25.141149 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.141113 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-w45x4"] Jul 29 00:37:25.164232 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.164204 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.170481 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.169205 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 29 00:37:25.170481 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.169480 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nzh86\"" Jul 29 00:37:25.170481 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.169819 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 29 00:37:25.170481 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.170019 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 29 00:37:25.170481 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.170279 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 29 00:37:25.246087 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.245674 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52e27182-467d-47b0-992a-7e86173ab584-root\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.246087 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.245732 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-accelerators-collector-config\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.246087 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.245790 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-288cc\" (UniqueName: \"kubernetes.io/projected/52e27182-467d-47b0-992a-7e86173ab584-kube-api-access-288cc\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.246087 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.245837 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-textfile\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.246087 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.245865 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.246087 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.245893 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-wtmp\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.246087 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.245920 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52e27182-467d-47b0-992a-7e86173ab584-sys\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.246087 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.245942 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-tls\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.246087 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.245966 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52e27182-467d-47b0-992a-7e86173ab584-metrics-client-ca\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.347857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.347055 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.347857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.347105 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-wtmp\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.347857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.347136 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52e27182-467d-47b0-992a-7e86173ab584-sys\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.347857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.347160 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-tls\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.347857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.347192 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52e27182-467d-47b0-992a-7e86173ab584-metrics-client-ca\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.347857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.347222 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52e27182-467d-47b0-992a-7e86173ab584-root\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.347857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.347257 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-accelerators-collector-config\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.347857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.347273 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52e27182-467d-47b0-992a-7e86173ab584-sys\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.347857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.347452 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-wtmp\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.347857 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.347546 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52e27182-467d-47b0-992a-7e86173ab584-root\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.347857 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:37:25.347624 2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 29 00:37:25.347857 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:37:25.347685 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-tls podName:52e27182-467d-47b0-992a-7e86173ab584 nodeName:}" failed. No retries permitted until 2026-07-29 00:37:25.847658264 +0000 UTC m=+97.506884141 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-tls") pod "node-exporter-w45x4" (UID: "52e27182-467d-47b0-992a-7e86173ab584") : secret "node-exporter-tls" not found Jul 29 00:37:25.349115 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.349021 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288cc\" (UniqueName: \"kubernetes.io/projected/52e27182-467d-47b0-992a-7e86173ab584-kube-api-access-288cc\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.349558 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.349502 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-textfile\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.350384 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.350341 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-accelerators-collector-config\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.350867 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.350829 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-textfile\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.351655 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.351617 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52e27182-467d-47b0-992a-7e86173ab584-metrics-client-ca\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.361960 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.361913 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.366516 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.366458 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-288cc\" (UniqueName: \"kubernetes.io/projected/52e27182-467d-47b0-992a-7e86173ab584-kube-api-access-288cc\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.712851 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.712120 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77879686d6-qmm2n"] Jul 29 00:37:25.854331 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.854273 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-tls\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:25.857361 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:25.857284 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52e27182-467d-47b0-992a-7e86173ab584-node-exporter-tls\") pod \"node-exporter-w45x4\" (UID: \"52e27182-467d-47b0-992a-7e86173ab584\") " pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:26.091339 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:26.091108 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w45x4" Jul 29 00:37:26.097779 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:26.097752 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2kc2k" Jul 29 00:37:26.353800 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:26.353708 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w45x4" event={"ID":"52e27182-467d-47b0-992a-7e86173ab584","Type":"ContainerStarted","Data":"bb61e1f164318c2f94792cefb45d490941b966f4953616158f6422f8ede934ff"} Jul 29 00:37:30.375292 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:30.375248 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wtq82" event={"ID":"a095e234-21c3-4383-ad01-d399215129a0","Type":"ContainerStarted","Data":"d0b3843e190448df4ce8f4ffe9cf952f10941dcfba7292c5923045a86e79e37b"} Jul 29 00:37:31.393409 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:31.393369 2558 generic.go:356] "Generic (PLEG): container finished" podID="52e27182-467d-47b0-992a-7e86173ab584" containerID="d1644ff8d27abd116c4274ac4aeebc03fa72e6fff236e5cca5ddbae611d86241" exitCode=0 Jul 29 00:37:31.393960 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:31.393471 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w45x4" event={"ID":"52e27182-467d-47b0-992a-7e86173ab584","Type":"ContainerDied","Data":"d1644ff8d27abd116c4274ac4aeebc03fa72e6fff236e5cca5ddbae611d86241"} Jul 29 00:37:32.401287 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:32.401250 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w45x4" event={"ID":"52e27182-467d-47b0-992a-7e86173ab584","Type":"ContainerStarted","Data":"2903b522a6cbf2292ebaa7afc82f498a5f24b6b6ba7770e5e2f140914522d005"} Jul 29 00:37:32.401765 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:32.401326 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w45x4" event={"ID":"52e27182-467d-47b0-992a-7e86173ab584","Type":"ContainerStarted","Data":"1d3b45023bf60a3e2ca6af1d0781294ab9ed79ba16e823f280a556e6b7c6c9a6"} Jul 29 00:37:32.420132 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:32.420084 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-w45x4" podStartSLOduration=2.821125492 podStartE2EDuration="7.420069457s" totalImagesPullingTime="4.502477117s" totalInitContainerRuntime="96.466848ms" isStatefulPod=true podCreationTimestamp="2026-07-29 00:37:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:32.419040976 +0000 UTC m=+104.078266870" watchObservedRunningTime="2026-07-29 00:37:32.420069457 +0000 UTC m=+104.079295328" Jul 29 00:37:35.414629 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:35.414589 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wtq82" event={"ID":"a095e234-21c3-4383-ad01-d399215129a0","Type":"ContainerStarted","Data":"6e11bc50260651ea267c9e7442ff9dccf888c1511f72bbdc98193c295504bbfb"} Jul 29 00:37:35.432077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:35.432024 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wtq82" podStartSLOduration=2.728988772 podStartE2EDuration="26.432005673s" totalImagesPullingTime="23.703016901s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:37:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:35.430930445 +0000 UTC m=+107.090156337" watchObservedRunningTime="2026-07-29 00:37:35.432005673 +0000 UTC m=+107.091231565" Jul 29 00:37:46.462132 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:46.462045 2558 generic.go:356] "Generic (PLEG): container finished" podID="ea6878e6-f9c1-4f52-8387-2c2c38d7084f" containerID="ca8d90ca22298924540df3e4c2d3bc6635f1f2cb80caa451f38a67dbd5104d2f" exitCode=0 Jul 29 00:37:46.462132 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:46.462089 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" event={"ID":"ea6878e6-f9c1-4f52-8387-2c2c38d7084f","Type":"ContainerDied","Data":"ca8d90ca22298924540df3e4c2d3bc6635f1f2cb80caa451f38a67dbd5104d2f"} Jul 29 00:37:46.462622 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:46.462430 2558 scope.go:117] "RemoveContainer" containerID="ca8d90ca22298924540df3e4c2d3bc6635f1f2cb80caa451f38a67dbd5104d2f" Jul 29 00:37:47.469920 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:47.469883 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb" event={"ID":"ea6878e6-f9c1-4f52-8387-2c2c38d7084f","Type":"ContainerStarted","Data":"4227e495499ef95fd73d1ce0edce6dcdb41633d1a666043052c9b4f9d936c94b"} Jul 29 00:37:50.748254 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:50.748208 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77879686d6-qmm2n" podUID="2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" containerName="registry" containerID="cri-o://04a0bd99ff26a0220e1269398143af0512b5f1dcf906ba1d68a709fcafbbe009" gracePeriod=30 Jul 29 00:37:51.040955 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.040927 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:37:51.186166 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.186134 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-certificates\") pod \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " Jul 29 00:37:51.186346 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.186174 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-trusted-ca\") pod \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " Jul 29 00:37:51.186346 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.186208 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-bound-sa-token\") pod \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " Jul 29 00:37:51.186346 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.186261 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-installation-pull-secrets\") pod \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " Jul 29 00:37:51.186346 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.186331 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-ca-trust-extracted\") pod \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " Jul 29 00:37:51.186549 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.186365 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls\") pod \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " Jul 29 00:37:51.186549 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.186409 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-image-registry-private-configuration\") pod \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " Jul 29 00:37:51.186549 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.186456 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26dmp\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-kube-api-access-26dmp\") pod \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\" (UID: \"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6\") " Jul 29 00:37:51.186698 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.186571 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" (UID: "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 00:37:51.186698 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.186656 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" (UID: "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 00:37:51.186847 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.186722 2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-certificates\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:37:51.186847 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.186742 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-trusted-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:37:51.188964 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.188930 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-kube-api-access-26dmp" (OuterVolumeSpecName: "kube-api-access-26dmp") pod "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" (UID: "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6"). InnerVolumeSpecName "kube-api-access-26dmp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 00:37:51.189084 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.188930 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" (UID: "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 00:37:51.189084 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.189005 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" (UID: "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 00:37:51.189203 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.189144 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" (UID: "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 00:37:51.189203 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.189157 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" (UID: "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 00:37:51.195276 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.195251 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" (UID: "2fa28369-78d4-43b6-bba0-1ea67ef2a6a6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 00:37:51.287998 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.287936 2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-ca-trust-extracted\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:37:51.287998 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.287963 2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-registry-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:37:51.287998 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.287975 2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-image-registry-private-configuration\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:37:51.287998 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.287999 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26dmp\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-kube-api-access-26dmp\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:37:51.288201 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.288009 2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-bound-sa-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:37:51.288201 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.288018 2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6-installation-pull-secrets\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:37:51.486229 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.486193 2558 generic.go:356] "Generic (PLEG): container finished" podID="2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" containerID="04a0bd99ff26a0220e1269398143af0512b5f1dcf906ba1d68a709fcafbbe009" exitCode=0 Jul 29 00:37:51.486407 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.486235 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77879686d6-qmm2n" Jul 29 00:37:51.486407 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.486239 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77879686d6-qmm2n" event={"ID":"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6","Type":"ContainerDied","Data":"04a0bd99ff26a0220e1269398143af0512b5f1dcf906ba1d68a709fcafbbe009"} Jul 29 00:37:51.486407 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.486392 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77879686d6-qmm2n" event={"ID":"2fa28369-78d4-43b6-bba0-1ea67ef2a6a6","Type":"ContainerDied","Data":"3e0594451135d4814bf7e4052e45242a0bd0c13f16f5dc23371a754aea4250d4"} Jul 29 00:37:51.486528 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.486413 2558 scope.go:117] "RemoveContainer" containerID="04a0bd99ff26a0220e1269398143af0512b5f1dcf906ba1d68a709fcafbbe009" Jul 29 00:37:51.495664 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.495646 2558 scope.go:117] "RemoveContainer" containerID="04a0bd99ff26a0220e1269398143af0512b5f1dcf906ba1d68a709fcafbbe009" Jul 29 00:37:51.495915 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:37:51.495891 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04a0bd99ff26a0220e1269398143af0512b5f1dcf906ba1d68a709fcafbbe009\": container with ID starting with 04a0bd99ff26a0220e1269398143af0512b5f1dcf906ba1d68a709fcafbbe009 not found: ID does not exist" containerID="04a0bd99ff26a0220e1269398143af0512b5f1dcf906ba1d68a709fcafbbe009" Jul 29 00:37:51.495952 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.495925 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04a0bd99ff26a0220e1269398143af0512b5f1dcf906ba1d68a709fcafbbe009"} err="failed to get container status \"04a0bd99ff26a0220e1269398143af0512b5f1dcf906ba1d68a709fcafbbe009\": rpc error: code = NotFound desc = could not find container \"04a0bd99ff26a0220e1269398143af0512b5f1dcf906ba1d68a709fcafbbe009\": container with ID starting with 04a0bd99ff26a0220e1269398143af0512b5f1dcf906ba1d68a709fcafbbe009 not found: ID does not exist" Jul 29 00:37:51.506629 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.506605 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77879686d6-qmm2n"] Jul 29 00:37:51.509801 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:51.509780 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77879686d6-qmm2n"] Jul 29 00:37:52.788177 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:37:52.788143 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" path="/var/lib/kubelet/pods/2fa28369-78d4-43b6-bba0-1ea67ef2a6a6/volumes" Jul 29 00:38:01.527258 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:38:01.527173 2558 generic.go:356] "Generic (PLEG): container finished" podID="711fe493-7a2b-41ca-b22e-a388bd42275a" containerID="1f08f7d53403c9bc2fe39ebb4598e3b5a0d86dba9720905fdc5a3b87a955df12" exitCode=0 Jul 29 00:38:01.527641 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:38:01.527251 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" event={"ID":"711fe493-7a2b-41ca-b22e-a388bd42275a","Type":"ContainerDied","Data":"1f08f7d53403c9bc2fe39ebb4598e3b5a0d86dba9720905fdc5a3b87a955df12"} Jul 29 00:38:01.527703 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:38:01.527684 2558 scope.go:117] "RemoveContainer" containerID="1f08f7d53403c9bc2fe39ebb4598e3b5a0d86dba9720905fdc5a3b87a955df12" Jul 29 00:38:02.533400 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:38:02.533361 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2lrbd" event={"ID":"711fe493-7a2b-41ca-b22e-a388bd42275a","Type":"ContainerStarted","Data":"f9d87f55293d938f00411cc1391de867adb5fa7d14e33aea106e3a266cedec24"} Jul 29 00:40:48.702874 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:48.702842 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 00:40:48.705068 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:48.705046 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 00:40:48.707779 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:48.707757 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 00:40:48.710081 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:48.710061 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 00:40:48.714707 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:48.714684 2558 kubelet.go:1648] "Image garbage collection succeeded" Jul 29 00:40:58.672809 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.672772 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj"] Jul 29 00:40:58.674239 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.673069 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" containerName="registry" Jul 29 00:40:58.674239 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.673080 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" containerName="registry" Jul 29 00:40:58.674239 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.673143 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fa28369-78d4-43b6-bba0-1ea67ef2a6a6" containerName="registry" Jul 29 00:40:58.675074 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.675057 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.678627 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.678602 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 29 00:40:58.678825 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.678803 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 29 00:40:58.679033 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.679014 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 29 00:40:58.679115 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.679048 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 29 00:40:58.679406 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.679388 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mbnrz\"" Jul 29 00:40:58.701872 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.701848 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj"] Jul 29 00:40:58.774938 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.774898 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/07ca9735-533c-46dc-be20-7decdb7be34a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.775157 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.775049 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/07ca9735-533c-46dc-be20-7decdb7be34a-manifests\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.775157 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.775084 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07ca9735-533c-46dc-be20-7decdb7be34a-tmp\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.775157 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.775103 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf47b\" (UniqueName: \"kubernetes.io/projected/07ca9735-533c-46dc-be20-7decdb7be34a-kube-api-access-gf47b\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.775157 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.775124 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07ca9735-533c-46dc-be20-7decdb7be34a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.875637 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.875597 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/07ca9735-533c-46dc-be20-7decdb7be34a-manifests\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.875637 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.875640 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07ca9735-533c-46dc-be20-7decdb7be34a-tmp\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.875884 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.875662 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf47b\" (UniqueName: \"kubernetes.io/projected/07ca9735-533c-46dc-be20-7decdb7be34a-kube-api-access-gf47b\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.875884 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.875692 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07ca9735-533c-46dc-be20-7decdb7be34a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.875884 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.875796 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/07ca9735-533c-46dc-be20-7decdb7be34a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.876084 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.876045 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/07ca9735-533c-46dc-be20-7decdb7be34a-manifests\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.878047 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.878024 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07ca9735-533c-46dc-be20-7decdb7be34a-tmp\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.878204 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.878185 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/07ca9735-533c-46dc-be20-7decdb7be34a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.878250 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.878187 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07ca9735-533c-46dc-be20-7decdb7be34a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.885652 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.885631 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf47b\" (UniqueName: \"kubernetes.io/projected/07ca9735-533c-46dc-be20-7decdb7be34a-kube-api-access-gf47b\") pod \"opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj\" (UID: \"07ca9735-533c-46dc-be20-7decdb7be34a\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:58.985067 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:58.984988 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:40:59.119683 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:59.119628 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj"] Jul 29 00:40:59.122967 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:40:59.122932 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ca9735_533c_46dc_be20_7decdb7be34a.slice/crio-c826adafe4d9831863415b9508b77d88a6bd7ec6f7d1ce7eada139d385db2e2b WatchSource:0}: Error finding container c826adafe4d9831863415b9508b77d88a6bd7ec6f7d1ce7eada139d385db2e2b: Status 404 returned error can't find the container with id c826adafe4d9831863415b9508b77d88a6bd7ec6f7d1ce7eada139d385db2e2b Jul 29 00:40:59.124706 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:59.124685 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 00:40:59.211515 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:40:59.211472 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" event={"ID":"07ca9735-533c-46dc-be20-7decdb7be34a","Type":"ContainerStarted","Data":"c826adafe4d9831863415b9508b77d88a6bd7ec6f7d1ce7eada139d385db2e2b"} Jul 29 00:41:03.232260 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:03.232217 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" event={"ID":"07ca9735-533c-46dc-be20-7decdb7be34a","Type":"ContainerStarted","Data":"45bc8b2b559bc1632f06cd83f6111ac10d0845e2a1bfc826dea995c6966d1bdf"} Jul 29 00:41:04.238222 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:04.238189 2558 generic.go:356] "Generic (PLEG): container finished" podID="07ca9735-533c-46dc-be20-7decdb7be34a" containerID="45bc8b2b559bc1632f06cd83f6111ac10d0845e2a1bfc826dea995c6966d1bdf" exitCode=0 Jul 29 00:41:04.238617 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:04.238250 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" event={"ID":"07ca9735-533c-46dc-be20-7decdb7be34a","Type":"ContainerDied","Data":"45bc8b2b559bc1632f06cd83f6111ac10d0845e2a1bfc826dea995c6966d1bdf"} Jul 29 00:41:05.244423 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:05.244384 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" event={"ID":"07ca9735-533c-46dc-be20-7decdb7be34a","Type":"ContainerStarted","Data":"8f5e7175524e557eb955b02a12edeb3f64610b024dc5da73f3c63ada72c01061"} Jul 29 00:41:05.244810 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:05.244471 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:41:05.289463 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:05.289404 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" podStartSLOduration=2.680675195 podStartE2EDuration="7.289383371s" totalImagesPullingTime="3.563211705s" totalInitContainerRuntime="1.045496471s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:40:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:41:05.288727921 +0000 UTC m=+316.947953825" watchObservedRunningTime="2026-07-29 00:41:05.289383371 +0000 UTC m=+316.948609277" Jul 29 00:41:16.251125 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:16.251090 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj" Jul 29 00:41:28.566887 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.566854 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-896f4944f-597sf"] Jul 29 00:41:28.573136 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.573116 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:28.575799 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.575771 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 29 00:41:28.575921 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.575870 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 29 00:41:28.576389 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.576367 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 29 00:41:28.576925 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.576409 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-pg7r9\"" Jul 29 00:41:28.576925 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.576442 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 29 00:41:28.577067 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.576613 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 29 00:41:28.578330 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.578297 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-896f4944f-597sf"] Jul 29 00:41:28.736330 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.736275 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f2bac32b-55cf-493d-94a8-6dd0c637d1e6-metrics-cert\") pod \"lws-controller-manager-896f4944f-597sf\" (UID: \"f2bac32b-55cf-493d-94a8-6dd0c637d1e6\") " pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:28.736512 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.736362 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2bac32b-55cf-493d-94a8-6dd0c637d1e6-cert\") pod \"lws-controller-manager-896f4944f-597sf\" (UID: \"f2bac32b-55cf-493d-94a8-6dd0c637d1e6\") " pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:28.736512 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.736382 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f2bac32b-55cf-493d-94a8-6dd0c637d1e6-manager-config\") pod \"lws-controller-manager-896f4944f-597sf\" (UID: \"f2bac32b-55cf-493d-94a8-6dd0c637d1e6\") " pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:28.736512 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.736397 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89xp6\" (UniqueName: \"kubernetes.io/projected/f2bac32b-55cf-493d-94a8-6dd0c637d1e6-kube-api-access-89xp6\") pod \"lws-controller-manager-896f4944f-597sf\" (UID: \"f2bac32b-55cf-493d-94a8-6dd0c637d1e6\") " pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:28.836882 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.836786 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2bac32b-55cf-493d-94a8-6dd0c637d1e6-cert\") pod \"lws-controller-manager-896f4944f-597sf\" (UID: \"f2bac32b-55cf-493d-94a8-6dd0c637d1e6\") " pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:28.836882 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.836834 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f2bac32b-55cf-493d-94a8-6dd0c637d1e6-manager-config\") pod \"lws-controller-manager-896f4944f-597sf\" (UID: \"f2bac32b-55cf-493d-94a8-6dd0c637d1e6\") " pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:28.836882 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.836866 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89xp6\" (UniqueName: \"kubernetes.io/projected/f2bac32b-55cf-493d-94a8-6dd0c637d1e6-kube-api-access-89xp6\") pod \"lws-controller-manager-896f4944f-597sf\" (UID: \"f2bac32b-55cf-493d-94a8-6dd0c637d1e6\") " pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:28.837171 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.836925 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f2bac32b-55cf-493d-94a8-6dd0c637d1e6-metrics-cert\") pod \"lws-controller-manager-896f4944f-597sf\" (UID: \"f2bac32b-55cf-493d-94a8-6dd0c637d1e6\") " pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:28.837894 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.837837 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f2bac32b-55cf-493d-94a8-6dd0c637d1e6-manager-config\") pod \"lws-controller-manager-896f4944f-597sf\" (UID: \"f2bac32b-55cf-493d-94a8-6dd0c637d1e6\") " pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:28.840245 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.840218 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2bac32b-55cf-493d-94a8-6dd0c637d1e6-cert\") pod \"lws-controller-manager-896f4944f-597sf\" (UID: \"f2bac32b-55cf-493d-94a8-6dd0c637d1e6\") " pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:28.840518 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.840494 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f2bac32b-55cf-493d-94a8-6dd0c637d1e6-metrics-cert\") pod \"lws-controller-manager-896f4944f-597sf\" (UID: \"f2bac32b-55cf-493d-94a8-6dd0c637d1e6\") " pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:28.846854 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.846825 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89xp6\" (UniqueName: \"kubernetes.io/projected/f2bac32b-55cf-493d-94a8-6dd0c637d1e6-kube-api-access-89xp6\") pod \"lws-controller-manager-896f4944f-597sf\" (UID: \"f2bac32b-55cf-493d-94a8-6dd0c637d1e6\") " pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:28.882757 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:28.882727 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:29.008449 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:29.008419 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-896f4944f-597sf"] Jul 29 00:41:29.012029 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:41:29.012004 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2bac32b_55cf_493d_94a8_6dd0c637d1e6.slice/crio-ccd8909c62ddd7fff63c9d177a60c31674e3a61f7c8bb40292e0e872db35fc5c WatchSource:0}: Error finding container ccd8909c62ddd7fff63c9d177a60c31674e3a61f7c8bb40292e0e872db35fc5c: Status 404 returned error can't find the container with id ccd8909c62ddd7fff63c9d177a60c31674e3a61f7c8bb40292e0e872db35fc5c Jul 29 00:41:29.344969 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:29.344935 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" event={"ID":"f2bac32b-55cf-493d-94a8-6dd0c637d1e6","Type":"ContainerStarted","Data":"ccd8909c62ddd7fff63c9d177a60c31674e3a61f7c8bb40292e0e872db35fc5c"} Jul 29 00:41:30.552540 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.552500 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h284l"] Jul 29 00:41:30.558822 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.558793 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9"] Jul 29 00:41:30.563521 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.560443 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h284l" Jul 29 00:41:30.564696 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.564668 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-tw86z\"" Jul 29 00:41:30.564821 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.564698 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 29 00:41:30.565797 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.565480 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9" Jul 29 00:41:30.568220 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.567588 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 29 00:41:30.568220 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.567821 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-fqhdq\"" Jul 29 00:41:30.570365 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.570125 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h284l"] Jul 29 00:41:30.572782 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.572735 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9"] Jul 29 00:41:30.650179 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.650146 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c87zh\" (UniqueName: \"kubernetes.io/projected/48d1fcf8-0b6f-4f98-958b-6cf70fb8939b-kube-api-access-c87zh\") pod \"kserve-controller-manager-d6f4b65b8-h284l\" (UID: \"48d1fcf8-0b6f-4f98-958b-6cf70fb8939b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h284l" Jul 29 00:41:30.650359 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.650221 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48d1fcf8-0b6f-4f98-958b-6cf70fb8939b-cert\") pod \"kserve-controller-manager-d6f4b65b8-h284l\" (UID: \"48d1fcf8-0b6f-4f98-958b-6cf70fb8939b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h284l" Jul 29 00:41:30.751479 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.751440 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f255234-9a38-4882-bd6b-95e2ccc46eb8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-64ss9\" (UID: \"3f255234-9a38-4882-bd6b-95e2ccc46eb8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9" Jul 29 00:41:30.751660 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.751497 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8jxq\" (UniqueName: \"kubernetes.io/projected/3f255234-9a38-4882-bd6b-95e2ccc46eb8-kube-api-access-r8jxq\") pod \"llmisvc-controller-manager-777c6bd7d5-64ss9\" (UID: \"3f255234-9a38-4882-bd6b-95e2ccc46eb8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9" Jul 29 00:41:30.751660 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.751594 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c87zh\" (UniqueName: \"kubernetes.io/projected/48d1fcf8-0b6f-4f98-958b-6cf70fb8939b-kube-api-access-c87zh\") pod \"kserve-controller-manager-d6f4b65b8-h284l\" (UID: \"48d1fcf8-0b6f-4f98-958b-6cf70fb8939b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h284l" Jul 29 00:41:30.751795 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.751666 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48d1fcf8-0b6f-4f98-958b-6cf70fb8939b-cert\") pod \"kserve-controller-manager-d6f4b65b8-h284l\" (UID: \"48d1fcf8-0b6f-4f98-958b-6cf70fb8939b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h284l" Jul 29 00:41:30.751859 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:41:30.751811 2558 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 29 00:41:30.751917 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:41:30.751871 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48d1fcf8-0b6f-4f98-958b-6cf70fb8939b-cert podName:48d1fcf8-0b6f-4f98-958b-6cf70fb8939b nodeName:}" failed. No retries permitted until 2026-07-29 00:41:31.251850163 +0000 UTC m=+342.911076039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/48d1fcf8-0b6f-4f98-958b-6cf70fb8939b-cert") pod "kserve-controller-manager-d6f4b65b8-h284l" (UID: "48d1fcf8-0b6f-4f98-958b-6cf70fb8939b") : secret "kserve-webhook-server-cert" not found Jul 29 00:41:30.766634 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.766593 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c87zh\" (UniqueName: \"kubernetes.io/projected/48d1fcf8-0b6f-4f98-958b-6cf70fb8939b-kube-api-access-c87zh\") pod \"kserve-controller-manager-d6f4b65b8-h284l\" (UID: \"48d1fcf8-0b6f-4f98-958b-6cf70fb8939b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h284l" Jul 29 00:41:30.852394 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.852358 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f255234-9a38-4882-bd6b-95e2ccc46eb8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-64ss9\" (UID: \"3f255234-9a38-4882-bd6b-95e2ccc46eb8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9" Jul 29 00:41:30.852596 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.852410 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8jxq\" (UniqueName: \"kubernetes.io/projected/3f255234-9a38-4882-bd6b-95e2ccc46eb8-kube-api-access-r8jxq\") pod \"llmisvc-controller-manager-777c6bd7d5-64ss9\" (UID: \"3f255234-9a38-4882-bd6b-95e2ccc46eb8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9" Jul 29 00:41:30.852672 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:41:30.852611 2558 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 29 00:41:30.852726 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:41:30.852681 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f255234-9a38-4882-bd6b-95e2ccc46eb8-cert podName:3f255234-9a38-4882-bd6b-95e2ccc46eb8 nodeName:}" failed. No retries permitted until 2026-07-29 00:41:31.352660414 +0000 UTC m=+343.011886289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3f255234-9a38-4882-bd6b-95e2ccc46eb8-cert") pod "llmisvc-controller-manager-777c6bd7d5-64ss9" (UID: "3f255234-9a38-4882-bd6b-95e2ccc46eb8") : secret "llmisvc-webhook-server-cert" not found Jul 29 00:41:30.865501 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:30.865465 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8jxq\" (UniqueName: \"kubernetes.io/projected/3f255234-9a38-4882-bd6b-95e2ccc46eb8-kube-api-access-r8jxq\") pod \"llmisvc-controller-manager-777c6bd7d5-64ss9\" (UID: \"3f255234-9a38-4882-bd6b-95e2ccc46eb8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9" Jul 29 00:41:31.256116 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:31.256028 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48d1fcf8-0b6f-4f98-958b-6cf70fb8939b-cert\") pod \"kserve-controller-manager-d6f4b65b8-h284l\" (UID: \"48d1fcf8-0b6f-4f98-958b-6cf70fb8939b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h284l" Jul 29 00:41:31.259037 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:31.259006 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48d1fcf8-0b6f-4f98-958b-6cf70fb8939b-cert\") pod \"kserve-controller-manager-d6f4b65b8-h284l\" (UID: \"48d1fcf8-0b6f-4f98-958b-6cf70fb8939b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h284l" Jul 29 00:41:31.356755 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:31.356724 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f255234-9a38-4882-bd6b-95e2ccc46eb8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-64ss9\" (UID: \"3f255234-9a38-4882-bd6b-95e2ccc46eb8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9" Jul 29 00:41:31.359349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:31.359324 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f255234-9a38-4882-bd6b-95e2ccc46eb8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-64ss9\" (UID: \"3f255234-9a38-4882-bd6b-95e2ccc46eb8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9" Jul 29 00:41:31.479719 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:31.479680 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h284l" Jul 29 00:41:31.489802 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:31.489774 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9" Jul 29 00:41:31.820483 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:41:31.820377 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48d1fcf8_0b6f_4f98_958b_6cf70fb8939b.slice/crio-916c7ddf749abc56befd6d16863d697bf3e1aa34bc32533dea5941d395b816db WatchSource:0}: Error finding container 916c7ddf749abc56befd6d16863d697bf3e1aa34bc32533dea5941d395b816db: Status 404 returned error can't find the container with id 916c7ddf749abc56befd6d16863d697bf3e1aa34bc32533dea5941d395b816db Jul 29 00:41:31.825412 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:31.825383 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h284l"] Jul 29 00:41:31.840262 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:31.840144 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9"] Jul 29 00:41:31.842698 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:41:31.842670 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3f255234_9a38_4882_bd6b_95e2ccc46eb8.slice/crio-31c2e843f89eb6f11f10e1c81ee68d98ffe5ba219fd74697f53eb2bc07262400 WatchSource:0}: Error finding container 31c2e843f89eb6f11f10e1c81ee68d98ffe5ba219fd74697f53eb2bc07262400: Status 404 returned error can't find the container with id 31c2e843f89eb6f11f10e1c81ee68d98ffe5ba219fd74697f53eb2bc07262400 Jul 29 00:41:32.361497 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:32.361456 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" event={"ID":"f2bac32b-55cf-493d-94a8-6dd0c637d1e6","Type":"ContainerStarted","Data":"10df3aa213092b9cdab91027e8226d5fae1c2a0b686a4c4cc27597e0b5fdd06f"} Jul 29 00:41:32.361693 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:32.361560 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:41:32.362758 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:32.362732 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h284l" event={"ID":"48d1fcf8-0b6f-4f98-958b-6cf70fb8939b","Type":"ContainerStarted","Data":"916c7ddf749abc56befd6d16863d697bf3e1aa34bc32533dea5941d395b816db"} Jul 29 00:41:32.363861 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:32.363842 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9" event={"ID":"3f255234-9a38-4882-bd6b-95e2ccc46eb8","Type":"ContainerStarted","Data":"31c2e843f89eb6f11f10e1c81ee68d98ffe5ba219fd74697f53eb2bc07262400"} Jul 29 00:41:32.391580 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:32.391538 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" podStartSLOduration=1.696393988 podStartE2EDuration="4.391525645s" totalImagesPullingTime="2.695131657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:41:32.390368845 +0000 UTC m=+344.049594740" watchObservedRunningTime="2026-07-29 00:41:32.391525645 +0000 UTC m=+344.050751537" Jul 29 00:41:34.375941 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:34.375860 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h284l" event={"ID":"48d1fcf8-0b6f-4f98-958b-6cf70fb8939b","Type":"ContainerStarted","Data":"bd8f1741d7488c8dd8d7050983d9f91ca1af385ad82bd1ba3bd0625df11fc131"} Jul 29 00:41:34.376296 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:34.375961 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h284l" Jul 29 00:41:34.402135 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:34.402081 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h284l" podStartSLOduration=2.123852885 podStartE2EDuration="4.40206517s" totalImagesPullingTime="2.278212285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:41:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:41:34.400091895 +0000 UTC m=+346.059317789" watchObservedRunningTime="2026-07-29 00:41:34.40206517 +0000 UTC m=+346.061291064" Jul 29 00:41:35.382156 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:35.382111 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9" event={"ID":"3f255234-9a38-4882-bd6b-95e2ccc46eb8","Type":"ContainerStarted","Data":"d59b1ec7da0ef0d6e9ff032e224788bda995154d132a1e64baa5dad7bec72594"} Jul 29 00:41:35.398887 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:35.398827 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9" podStartSLOduration=2.208027006 podStartE2EDuration="5.398812764s" totalImagesPullingTime="3.190785758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:41:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:41:35.397479258 +0000 UTC m=+347.056705145" watchObservedRunningTime="2026-07-29 00:41:35.398812764 +0000 UTC m=+347.058038657" Jul 29 00:41:36.386208 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:36.386173 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9" Jul 29 00:41:43.371782 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:41:43.371701 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-896f4944f-597sf" Jul 29 00:42:05.387633 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:42:05.387601 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h284l" Jul 29 00:42:07.392863 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:42:07.392835 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64ss9" Jul 29 00:43:14.164501 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.164423 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4"] Jul 29 00:43:14.166607 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.166588 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:14.168849 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.168827 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 29 00:43:14.168957 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.168937 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sq5mr\"" Jul 29 00:43:14.169004 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.168991 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 29 00:43:14.183928 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.183907 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4"] Jul 29 00:43:14.308151 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.308114 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z58hn\" (UniqueName: \"kubernetes.io/projected/4eb1ed11-d9af-4fea-8430-2ea48966cb19-kube-api-access-z58hn\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-c9xf4\" (UID: \"4eb1ed11-d9af-4fea-8430-2ea48966cb19\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:14.308332 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.308169 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4eb1ed11-d9af-4fea-8430-2ea48966cb19-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-c9xf4\" (UID: \"4eb1ed11-d9af-4fea-8430-2ea48966cb19\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:14.319013 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.318973 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4"] Jul 29 00:43:14.319298 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:43:14.319272 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-z58hn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" podUID="4eb1ed11-d9af-4fea-8430-2ea48966cb19" Jul 29 00:43:14.408604 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.408570 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z58hn\" (UniqueName: \"kubernetes.io/projected/4eb1ed11-d9af-4fea-8430-2ea48966cb19-kube-api-access-z58hn\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-c9xf4\" (UID: \"4eb1ed11-d9af-4fea-8430-2ea48966cb19\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:14.408604 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.408611 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4eb1ed11-d9af-4fea-8430-2ea48966cb19-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-c9xf4\" (UID: \"4eb1ed11-d9af-4fea-8430-2ea48966cb19\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:14.409020 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.409000 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4eb1ed11-d9af-4fea-8430-2ea48966cb19-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-c9xf4\" (UID: \"4eb1ed11-d9af-4fea-8430-2ea48966cb19\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:14.416782 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.416725 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z58hn\" (UniqueName: \"kubernetes.io/projected/4eb1ed11-d9af-4fea-8430-2ea48966cb19-kube-api-access-z58hn\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-c9xf4\" (UID: \"4eb1ed11-d9af-4fea-8430-2ea48966cb19\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:14.808037 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.807951 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4"] Jul 29 00:43:14.811713 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.811685 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4"] Jul 29 00:43:14.813961 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.813937 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:14.815643 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:43:14.815614 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-c9xf4\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="4eb1ed11-d9af-4fea-8430-2ea48966cb19" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:14.821112 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.821094 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:14.823424 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:43:14.823399 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-c9xf4\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="4eb1ed11-d9af-4fea-8430-2ea48966cb19" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:14.863953 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.863928 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g"] Jul 29 00:43:14.866285 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.866261 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" Jul 29 00:43:14.872028 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:43:14.871996 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-c9xf4\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="4eb1ed11-d9af-4fea-8430-2ea48966cb19" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:14.889826 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.889797 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g"] Jul 29 00:43:14.912406 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.912382 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z58hn\" (UniqueName: \"kubernetes.io/projected/4eb1ed11-d9af-4fea-8430-2ea48966cb19-kube-api-access-z58hn\") pod \"4eb1ed11-d9af-4fea-8430-2ea48966cb19\" (UID: \"4eb1ed11-d9af-4fea-8430-2ea48966cb19\") " Jul 29 00:43:14.912798 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.912775 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4eb1ed11-d9af-4fea-8430-2ea48966cb19-extensions-socket-volume\") pod \"4eb1ed11-d9af-4fea-8430-2ea48966cb19\" (UID: \"4eb1ed11-d9af-4fea-8430-2ea48966cb19\") " Jul 29 00:43:14.913083 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.913058 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eb1ed11-d9af-4fea-8430-2ea48966cb19-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4eb1ed11-d9af-4fea-8430-2ea48966cb19" (UID: "4eb1ed11-d9af-4fea-8430-2ea48966cb19"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 00:43:14.914606 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:14.914584 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eb1ed11-d9af-4fea-8430-2ea48966cb19-kube-api-access-z58hn" (OuterVolumeSpecName: "kube-api-access-z58hn") pod "4eb1ed11-d9af-4fea-8430-2ea48966cb19" (UID: "4eb1ed11-d9af-4fea-8430-2ea48966cb19"). InnerVolumeSpecName "kube-api-access-z58hn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 00:43:15.013564 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:15.013526 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da9f0e12-d167-4a56-a326-46842790a675-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j676g\" (UID: \"da9f0e12-d167-4a56-a326-46842790a675\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" Jul 29 00:43:15.013732 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:15.013609 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwh6w\" (UniqueName: \"kubernetes.io/projected/da9f0e12-d167-4a56-a326-46842790a675-kube-api-access-kwh6w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j676g\" (UID: \"da9f0e12-d167-4a56-a326-46842790a675\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" Jul 29 00:43:15.013732 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:15.013659 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4eb1ed11-d9af-4fea-8430-2ea48966cb19-extensions-socket-volume\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:43:15.013732 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:15.013669 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z58hn\" (UniqueName: \"kubernetes.io/projected/4eb1ed11-d9af-4fea-8430-2ea48966cb19-kube-api-access-z58hn\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:43:15.114537 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:15.114503 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da9f0e12-d167-4a56-a326-46842790a675-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j676g\" (UID: \"da9f0e12-d167-4a56-a326-46842790a675\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" Jul 29 00:43:15.114692 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:15.114567 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwh6w\" (UniqueName: \"kubernetes.io/projected/da9f0e12-d167-4a56-a326-46842790a675-kube-api-access-kwh6w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j676g\" (UID: \"da9f0e12-d167-4a56-a326-46842790a675\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" Jul 29 00:43:15.114867 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:15.114846 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da9f0e12-d167-4a56-a326-46842790a675-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j676g\" (UID: \"da9f0e12-d167-4a56-a326-46842790a675\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" Jul 29 00:43:15.124517 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:15.124491 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwh6w\" (UniqueName: \"kubernetes.io/projected/da9f0e12-d167-4a56-a326-46842790a675-kube-api-access-kwh6w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j676g\" (UID: \"da9f0e12-d167-4a56-a326-46842790a675\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" Jul 29 00:43:15.177200 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:15.177173 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" Jul 29 00:43:15.331318 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:15.331277 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g"] Jul 29 00:43:15.333872 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:43:15.333848 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda9f0e12_d167_4a56_a326_46842790a675.slice/crio-23ae5a4ffeceaf1b833685c5dd22af3973df9b67ac9d06b890cb09dffd91687d WatchSource:0}: Error finding container 23ae5a4ffeceaf1b833685c5dd22af3973df9b67ac9d06b890cb09dffd91687d: Status 404 returned error can't find the container with id 23ae5a4ffeceaf1b833685c5dd22af3973df9b67ac9d06b890cb09dffd91687d Jul 29 00:43:15.819325 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:15.819272 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" event={"ID":"da9f0e12-d167-4a56-a326-46842790a675","Type":"ContainerStarted","Data":"23ae5a4ffeceaf1b833685c5dd22af3973df9b67ac9d06b890cb09dffd91687d"} Jul 29 00:43:15.819482 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:15.819286 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:15.821211 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:43:15.821177 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-c9xf4\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="4eb1ed11-d9af-4fea-8430-2ea48966cb19" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:15.827124 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:43:15.827089 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-c9xf4\" is forbidden: User \"system:node:ip-10-0-135-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-11.ec2.internal' and this object" podUID="4eb1ed11-d9af-4fea-8430-2ea48966cb19" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-c9xf4" Jul 29 00:43:16.790255 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:16.790217 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eb1ed11-d9af-4fea-8430-2ea48966cb19" path="/var/lib/kubelet/pods/4eb1ed11-d9af-4fea-8430-2ea48966cb19/volumes" Jul 29 00:43:19.840639 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:19.840610 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" event={"ID":"da9f0e12-d167-4a56-a326-46842790a675","Type":"ContainerStarted","Data":"872a64cc3fc7bc766e835022bb00690eeee85920e88aa0fa93a082b767ad8735"} Jul 29 00:43:19.841071 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:19.840678 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" Jul 29 00:43:19.858507 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:19.858449 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" podStartSLOduration=1.854474639 podStartE2EDuration="5.858429903s" totalImagesPullingTime="4.003955264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:43:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:43:19.856328316 +0000 UTC m=+451.515554206" watchObservedRunningTime="2026-07-29 00:43:19.858429903 +0000 UTC m=+451.517655797" Jul 29 00:43:30.848182 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:30.848149 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" Jul 29 00:43:41.337607 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.337573 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g"] Jul 29 00:43:41.338094 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.337763 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" podUID="da9f0e12-d167-4a56-a326-46842790a675" containerName="manager" containerID="cri-o://872a64cc3fc7bc766e835022bb00690eeee85920e88aa0fa93a082b767ad8735" gracePeriod=10 Jul 29 00:43:41.587282 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.587256 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" Jul 29 00:43:41.732172 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.732098 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da9f0e12-d167-4a56-a326-46842790a675-extensions-socket-volume\") pod \"da9f0e12-d167-4a56-a326-46842790a675\" (UID: \"da9f0e12-d167-4a56-a326-46842790a675\") " Jul 29 00:43:41.732341 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.732183 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwh6w\" (UniqueName: \"kubernetes.io/projected/da9f0e12-d167-4a56-a326-46842790a675-kube-api-access-kwh6w\") pod \"da9f0e12-d167-4a56-a326-46842790a675\" (UID: \"da9f0e12-d167-4a56-a326-46842790a675\") " Jul 29 00:43:41.732651 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.732618 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da9f0e12-d167-4a56-a326-46842790a675-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "da9f0e12-d167-4a56-a326-46842790a675" (UID: "da9f0e12-d167-4a56-a326-46842790a675"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 00:43:41.734286 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.734266 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da9f0e12-d167-4a56-a326-46842790a675-kube-api-access-kwh6w" (OuterVolumeSpecName: "kube-api-access-kwh6w") pod "da9f0e12-d167-4a56-a326-46842790a675" (UID: "da9f0e12-d167-4a56-a326-46842790a675"). InnerVolumeSpecName "kube-api-access-kwh6w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 00:43:41.833460 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.833431 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da9f0e12-d167-4a56-a326-46842790a675-extensions-socket-volume\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:43:41.833460 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.833455 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwh6w\" (UniqueName: \"kubernetes.io/projected/da9f0e12-d167-4a56-a326-46842790a675-kube-api-access-kwh6w\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:43:41.937237 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.937202 2558 generic.go:356] "Generic (PLEG): container finished" podID="da9f0e12-d167-4a56-a326-46842790a675" containerID="872a64cc3fc7bc766e835022bb00690eeee85920e88aa0fa93a082b767ad8735" exitCode=0 Jul 29 00:43:41.937408 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.937243 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" Jul 29 00:43:41.937408 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.937291 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" event={"ID":"da9f0e12-d167-4a56-a326-46842790a675","Type":"ContainerDied","Data":"872a64cc3fc7bc766e835022bb00690eeee85920e88aa0fa93a082b767ad8735"} Jul 29 00:43:41.937408 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.937354 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g" event={"ID":"da9f0e12-d167-4a56-a326-46842790a675","Type":"ContainerDied","Data":"23ae5a4ffeceaf1b833685c5dd22af3973df9b67ac9d06b890cb09dffd91687d"} Jul 29 00:43:41.937408 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.937386 2558 scope.go:117] "RemoveContainer" containerID="872a64cc3fc7bc766e835022bb00690eeee85920e88aa0fa93a082b767ad8735" Jul 29 00:43:41.946039 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.946021 2558 scope.go:117] "RemoveContainer" containerID="872a64cc3fc7bc766e835022bb00690eeee85920e88aa0fa93a082b767ad8735" Jul 29 00:43:41.946285 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:43:41.946263 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"872a64cc3fc7bc766e835022bb00690eeee85920e88aa0fa93a082b767ad8735\": container with ID starting with 872a64cc3fc7bc766e835022bb00690eeee85920e88aa0fa93a082b767ad8735 not found: ID does not exist" containerID="872a64cc3fc7bc766e835022bb00690eeee85920e88aa0fa93a082b767ad8735" Jul 29 00:43:41.946411 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.946297 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"872a64cc3fc7bc766e835022bb00690eeee85920e88aa0fa93a082b767ad8735"} err="failed to get container status \"872a64cc3fc7bc766e835022bb00690eeee85920e88aa0fa93a082b767ad8735\": rpc error: code = NotFound desc = could not find container \"872a64cc3fc7bc766e835022bb00690eeee85920e88aa0fa93a082b767ad8735\": container with ID starting with 872a64cc3fc7bc766e835022bb00690eeee85920e88aa0fa93a082b767ad8735 not found: ID does not exist" Jul 29 00:43:41.957752 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.957731 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g"] Jul 29 00:43:41.961323 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:41.961285 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j676g"] Jul 29 00:43:42.789202 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:42.789168 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da9f0e12-d167-4a56-a326-46842790a675" path="/var/lib/kubelet/pods/da9f0e12-d167-4a56-a326-46842790a675/volumes" Jul 29 00:43:57.625419 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.625388 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm"] Jul 29 00:43:57.625884 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.625730 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da9f0e12-d167-4a56-a326-46842790a675" containerName="manager" Jul 29 00:43:57.625884 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.625741 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="da9f0e12-d167-4a56-a326-46842790a675" containerName="manager" Jul 29 00:43:57.625884 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.625815 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="da9f0e12-d167-4a56-a326-46842790a675" containerName="manager" Jul 29 00:43:57.629743 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.629722 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.631802 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.631778 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 29 00:43:57.632159 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.632144 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-nj946\"" Jul 29 00:43:57.638345 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.638321 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42z2t\" (UniqueName: \"kubernetes.io/projected/fb6b0154-70f0-474e-9bb5-b6f216183ba7-kube-api-access-42z2t\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.638460 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.638393 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.638460 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.638442 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.638576 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.638497 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.638629 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.638588 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.638682 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.638628 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.638682 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.638656 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.638761 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.638688 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.638761 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.638741 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.642476 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.642454 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm"] Jul 29 00:43:57.739658 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.739628 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42z2t\" (UniqueName: \"kubernetes.io/projected/fb6b0154-70f0-474e-9bb5-b6f216183ba7-kube-api-access-42z2t\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.739815 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.739669 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.739815 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.739700 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.739815 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.739730 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.739815 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.739787 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.740021 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.739818 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.740021 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.739844 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.740021 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.739880 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.740021 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.739910 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.740167 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.740126 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.740283 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.740260 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.740394 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.740375 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.740445 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.740427 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.740486 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.740457 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.742233 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.742203 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.742339 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.742324 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.758739 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.758715 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb6b0154-70f0-474e-9bb5-b6f216183ba7-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.759011 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.758994 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42z2t\" (UniqueName: \"kubernetes.io/projected/fb6b0154-70f0-474e-9bb5-b6f216183ba7-kube-api-access-42z2t\") pod \"maas-default-gateway-openshift-default-5ccb968c66-l5dkm\" (UID: \"fb6b0154-70f0-474e-9bb5-b6f216183ba7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:57.940025 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:57.939952 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:43:58.071201 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:58.071176 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm"] Jul 29 00:43:58.073349 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:43:58.073318 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb6b0154_70f0_474e_9bb5_b6f216183ba7.slice/crio-4ab3f55e0e982e99d57cf9eca4d58785a38f02d071a481f57184f55bc94b8b69 WatchSource:0}: Error finding container 4ab3f55e0e982e99d57cf9eca4d58785a38f02d071a481f57184f55bc94b8b69: Status 404 returned error can't find the container with id 4ab3f55e0e982e99d57cf9eca4d58785a38f02d071a481f57184f55bc94b8b69 Jul 29 00:43:59.017851 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:43:59.017766 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" event={"ID":"fb6b0154-70f0-474e-9bb5-b6f216183ba7","Type":"ContainerStarted","Data":"4ab3f55e0e982e99d57cf9eca4d58785a38f02d071a481f57184f55bc94b8b69"} Jul 29 00:44:00.676231 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:00.676181 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 00:44:00.676538 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:00.676281 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 00:44:00.676538 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:00.676347 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 00:44:01.028248 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:01.028161 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" event={"ID":"fb6b0154-70f0-474e-9bb5-b6f216183ba7","Type":"ContainerStarted","Data":"e3e25ad491dedff300a5b01858257804402f08d731e4c9782e642e29730db4d5"} Jul 29 00:44:01.049358 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:01.049293 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" podStartSLOduration=1.4486400640000001 podStartE2EDuration="4.049279867s" totalImagesPullingTime="2.600639803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:43:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:44:01.047905728 +0000 UTC m=+492.707131614" watchObservedRunningTime="2026-07-29 00:44:01.049279867 +0000 UTC m=+492.708505760" Jul 29 00:44:01.940896 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:01.940851 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:44:01.945601 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:01.945574 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:44:02.019592 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.019546 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zshj2"] Jul 29 00:44:02.021996 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.021972 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" Jul 29 00:44:02.023799 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.023776 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 29 00:44:02.023904 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.023886 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 29 00:44:02.023983 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.023970 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-frtpm\"" Jul 29 00:44:02.024105 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.024089 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 29 00:44:02.032257 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.032223 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zshj2"] Jul 29 00:44:02.033531 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.033508 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:44:02.034640 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.034619 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:44:02.084139 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.084102 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgr2n\" (UniqueName: \"kubernetes.io/projected/932b48ad-4bac-4974-b220-46bc1e7c9be8-kube-api-access-jgr2n\") pod \"limitador-limitador-64c94fd64f-zshj2\" (UID: \"932b48ad-4bac-4974-b220-46bc1e7c9be8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" Jul 29 00:44:02.084403 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.084380 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/932b48ad-4bac-4974-b220-46bc1e7c9be8-config-file\") pod \"limitador-limitador-64c94fd64f-zshj2\" (UID: \"932b48ad-4bac-4974-b220-46bc1e7c9be8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" Jul 29 00:44:02.120113 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.120076 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zshj2"] Jul 29 00:44:02.185725 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.185679 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgr2n\" (UniqueName: \"kubernetes.io/projected/932b48ad-4bac-4974-b220-46bc1e7c9be8-kube-api-access-jgr2n\") pod \"limitador-limitador-64c94fd64f-zshj2\" (UID: \"932b48ad-4bac-4974-b220-46bc1e7c9be8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" Jul 29 00:44:02.185725 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.185728 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/932b48ad-4bac-4974-b220-46bc1e7c9be8-config-file\") pod \"limitador-limitador-64c94fd64f-zshj2\" (UID: \"932b48ad-4bac-4974-b220-46bc1e7c9be8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" Jul 29 00:44:02.186421 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.186388 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/932b48ad-4bac-4974-b220-46bc1e7c9be8-config-file\") pod \"limitador-limitador-64c94fd64f-zshj2\" (UID: \"932b48ad-4bac-4974-b220-46bc1e7c9be8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" Jul 29 00:44:02.193759 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.193701 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgr2n\" (UniqueName: \"kubernetes.io/projected/932b48ad-4bac-4974-b220-46bc1e7c9be8-kube-api-access-jgr2n\") pod \"limitador-limitador-64c94fd64f-zshj2\" (UID: \"932b48ad-4bac-4974-b220-46bc1e7c9be8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" Jul 29 00:44:02.333076 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.333031 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" Jul 29 00:44:02.468456 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:02.468266 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zshj2"] Jul 29 00:44:02.471138 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:44:02.471110 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod932b48ad_4bac_4974_b220_46bc1e7c9be8.slice/crio-72731b496a7f86c4bd977f949fb5d3a5117e899e96fbb2ba32e6705dca977059 WatchSource:0}: Error finding container 72731b496a7f86c4bd977f949fb5d3a5117e899e96fbb2ba32e6705dca977059: Status 404 returned error can't find the container with id 72731b496a7f86c4bd977f949fb5d3a5117e899e96fbb2ba32e6705dca977059 Jul 29 00:44:03.038289 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:03.038251 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" event={"ID":"932b48ad-4bac-4974-b220-46bc1e7c9be8","Type":"ContainerStarted","Data":"72731b496a7f86c4bd977f949fb5d3a5117e899e96fbb2ba32e6705dca977059"} Jul 29 00:44:06.054234 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:06.054194 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" event={"ID":"932b48ad-4bac-4974-b220-46bc1e7c9be8","Type":"ContainerStarted","Data":"4b9f385dbba8a2e8444f92b2ca117eabd5913e333b6e83a795a0c6278ae0e71a"} Jul 29 00:44:06.054752 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:06.054264 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" Jul 29 00:44:06.070576 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:06.070535 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" podStartSLOduration=1.536607873 podStartE2EDuration="4.070520261s" totalImagesPullingTime="2.533912388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:44:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:44:06.070497303 +0000 UTC m=+497.729723199" watchObservedRunningTime="2026-07-29 00:44:06.070520261 +0000 UTC m=+497.729746156" Jul 29 00:44:17.060540 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:17.060503 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" Jul 29 00:44:18.519876 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:18.519835 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zshj2"] Jul 29 00:44:18.520356 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:18.520054 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" podUID="932b48ad-4bac-4974-b220-46bc1e7c9be8" containerName="limitador" containerID="cri-o://4b9f385dbba8a2e8444f92b2ca117eabd5913e333b6e83a795a0c6278ae0e71a" gracePeriod=30 Jul 29 00:44:19.061067 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.061042 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" Jul 29 00:44:19.110767 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.110692 2558 generic.go:356] "Generic (PLEG): container finished" podID="932b48ad-4bac-4974-b220-46bc1e7c9be8" containerID="4b9f385dbba8a2e8444f92b2ca117eabd5913e333b6e83a795a0c6278ae0e71a" exitCode=0 Jul 29 00:44:19.110767 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.110736 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" Jul 29 00:44:19.110767 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.110745 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" event={"ID":"932b48ad-4bac-4974-b220-46bc1e7c9be8","Type":"ContainerDied","Data":"4b9f385dbba8a2e8444f92b2ca117eabd5913e333b6e83a795a0c6278ae0e71a"} Jul 29 00:44:19.110960 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.110819 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zshj2" event={"ID":"932b48ad-4bac-4974-b220-46bc1e7c9be8","Type":"ContainerDied","Data":"72731b496a7f86c4bd977f949fb5d3a5117e899e96fbb2ba32e6705dca977059"} Jul 29 00:44:19.110960 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.110842 2558 scope.go:117] "RemoveContainer" containerID="4b9f385dbba8a2e8444f92b2ca117eabd5913e333b6e83a795a0c6278ae0e71a" Jul 29 00:44:19.118819 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.118802 2558 scope.go:117] "RemoveContainer" containerID="4b9f385dbba8a2e8444f92b2ca117eabd5913e333b6e83a795a0c6278ae0e71a" Jul 29 00:44:19.119065 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:44:19.119047 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b9f385dbba8a2e8444f92b2ca117eabd5913e333b6e83a795a0c6278ae0e71a\": container with ID starting with 4b9f385dbba8a2e8444f92b2ca117eabd5913e333b6e83a795a0c6278ae0e71a not found: ID does not exist" containerID="4b9f385dbba8a2e8444f92b2ca117eabd5913e333b6e83a795a0c6278ae0e71a" Jul 29 00:44:19.119128 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.119078 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b9f385dbba8a2e8444f92b2ca117eabd5913e333b6e83a795a0c6278ae0e71a"} err="failed to get container status \"4b9f385dbba8a2e8444f92b2ca117eabd5913e333b6e83a795a0c6278ae0e71a\": rpc error: code = NotFound desc = could not find container \"4b9f385dbba8a2e8444f92b2ca117eabd5913e333b6e83a795a0c6278ae0e71a\": container with ID starting with 4b9f385dbba8a2e8444f92b2ca117eabd5913e333b6e83a795a0c6278ae0e71a not found: ID does not exist" Jul 29 00:44:19.120284 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.120268 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgr2n\" (UniqueName: \"kubernetes.io/projected/932b48ad-4bac-4974-b220-46bc1e7c9be8-kube-api-access-jgr2n\") pod \"932b48ad-4bac-4974-b220-46bc1e7c9be8\" (UID: \"932b48ad-4bac-4974-b220-46bc1e7c9be8\") " Jul 29 00:44:19.120388 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.120332 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/932b48ad-4bac-4974-b220-46bc1e7c9be8-config-file\") pod \"932b48ad-4bac-4974-b220-46bc1e7c9be8\" (UID: \"932b48ad-4bac-4974-b220-46bc1e7c9be8\") " Jul 29 00:44:19.120694 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.120674 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/932b48ad-4bac-4974-b220-46bc1e7c9be8-config-file" (OuterVolumeSpecName: "config-file") pod "932b48ad-4bac-4974-b220-46bc1e7c9be8" (UID: "932b48ad-4bac-4974-b220-46bc1e7c9be8"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 00:44:19.122129 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.122103 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/932b48ad-4bac-4974-b220-46bc1e7c9be8-kube-api-access-jgr2n" (OuterVolumeSpecName: "kube-api-access-jgr2n") pod "932b48ad-4bac-4974-b220-46bc1e7c9be8" (UID: "932b48ad-4bac-4974-b220-46bc1e7c9be8"). InnerVolumeSpecName "kube-api-access-jgr2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 00:44:19.221362 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.221331 2558 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/932b48ad-4bac-4974-b220-46bc1e7c9be8-config-file\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:44:19.221362 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.221356 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgr2n\" (UniqueName: \"kubernetes.io/projected/932b48ad-4bac-4974-b220-46bc1e7c9be8-kube-api-access-jgr2n\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:44:19.430722 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.430688 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zshj2"] Jul 29 00:44:19.434473 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:19.434448 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zshj2"] Jul 29 00:44:20.788487 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:20.788455 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="932b48ad-4bac-4974-b220-46bc1e7c9be8" path="/var/lib/kubelet/pods/932b48ad-4bac-4974-b220-46bc1e7c9be8/volumes" Jul 29 00:44:23.685349 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.685292 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-k8868"] Jul 29 00:44:23.685775 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.685631 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="932b48ad-4bac-4974-b220-46bc1e7c9be8" containerName="limitador" Jul 29 00:44:23.685775 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.685644 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="932b48ad-4bac-4974-b220-46bc1e7c9be8" containerName="limitador" Jul 29 00:44:23.685775 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.685707 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="932b48ad-4bac-4974-b220-46bc1e7c9be8" containerName="limitador" Jul 29 00:44:23.690004 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.689985 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-k8868" Jul 29 00:44:23.691968 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.691940 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-ntpt4\"" Jul 29 00:44:23.692085 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.691942 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 29 00:44:23.692085 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.691988 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 29 00:44:23.692530 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.692514 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 29 00:44:23.699163 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.699142 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-k8868"] Jul 29 00:44:23.753652 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.753621 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f1cb9de5-a60c-4681-b089-f859b6c6b0ee-data\") pod \"postgres-69fb4594cf-k8868\" (UID: \"f1cb9de5-a60c-4681-b089-f859b6c6b0ee\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-k8868" Jul 29 00:44:23.753793 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.753661 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdp5m\" (UniqueName: \"kubernetes.io/projected/f1cb9de5-a60c-4681-b089-f859b6c6b0ee-kube-api-access-qdp5m\") pod \"postgres-69fb4594cf-k8868\" (UID: \"f1cb9de5-a60c-4681-b089-f859b6c6b0ee\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-k8868" Jul 29 00:44:23.855397 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.855364 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f1cb9de5-a60c-4681-b089-f859b6c6b0ee-data\") pod \"postgres-69fb4594cf-k8868\" (UID: \"f1cb9de5-a60c-4681-b089-f859b6c6b0ee\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-k8868" Jul 29 00:44:23.855592 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.855412 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdp5m\" (UniqueName: \"kubernetes.io/projected/f1cb9de5-a60c-4681-b089-f859b6c6b0ee-kube-api-access-qdp5m\") pod \"postgres-69fb4594cf-k8868\" (UID: \"f1cb9de5-a60c-4681-b089-f859b6c6b0ee\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-k8868" Jul 29 00:44:23.855814 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.855791 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f1cb9de5-a60c-4681-b089-f859b6c6b0ee-data\") pod \"postgres-69fb4594cf-k8868\" (UID: \"f1cb9de5-a60c-4681-b089-f859b6c6b0ee\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-k8868" Jul 29 00:44:23.863272 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:23.863248 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdp5m\" (UniqueName: \"kubernetes.io/projected/f1cb9de5-a60c-4681-b089-f859b6c6b0ee-kube-api-access-qdp5m\") pod \"postgres-69fb4594cf-k8868\" (UID: \"f1cb9de5-a60c-4681-b089-f859b6c6b0ee\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-k8868" Jul 29 00:44:24.001393 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:24.001303 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-k8868" Jul 29 00:44:24.126697 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:24.126663 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-k8868"] Jul 29 00:44:24.130026 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:44:24.129994 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1cb9de5_a60c_4681_b089_f859b6c6b0ee.slice/crio-87dfbc56385383f5dfa675bca10a219db276c437bfd329bfddf3c56df8f80fec WatchSource:0}: Error finding container 87dfbc56385383f5dfa675bca10a219db276c437bfd329bfddf3c56df8f80fec: Status 404 returned error can't find the container with id 87dfbc56385383f5dfa675bca10a219db276c437bfd329bfddf3c56df8f80fec Jul 29 00:44:25.142576 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:25.142483 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-k8868" event={"ID":"f1cb9de5-a60c-4681-b089-f859b6c6b0ee","Type":"ContainerStarted","Data":"87dfbc56385383f5dfa675bca10a219db276c437bfd329bfddf3c56df8f80fec"} Jul 29 00:44:29.163985 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:29.163952 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-k8868" event={"ID":"f1cb9de5-a60c-4681-b089-f859b6c6b0ee","Type":"ContainerStarted","Data":"32a1ff93b8c5f7ece0bd128269bdba5abdbac7814c257faf7bed709d2b010028"} Jul 29 00:44:29.164438 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:29.164053 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-k8868" Jul 29 00:44:29.179968 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:29.179918 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-k8868" podStartSLOduration=1.31644698 podStartE2EDuration="6.179901901s" totalImagesPullingTime="4.863454921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:44:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:44:29.179520296 +0000 UTC m=+520.838746188" watchObservedRunningTime="2026-07-29 00:44:29.179901901 +0000 UTC m=+520.839127794" Jul 29 00:44:35.196567 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:35.196535 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-k8868" Jul 29 00:44:42.330193 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:42.330107 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-787f676965-fs674"] Jul 29 00:44:42.337117 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:42.337095 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-787f676965-fs674" Jul 29 00:44:42.341521 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:42.341500 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 29 00:44:42.341750 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:42.341731 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xnhp8\"" Jul 29 00:44:42.341850 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:42.341834 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 29 00:44:42.359592 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:42.359566 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-787f676965-fs674"] Jul 29 00:44:42.413095 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:42.413064 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm8fk\" (UniqueName: \"kubernetes.io/projected/e89e5261-15f6-4685-8240-79677d98527c-kube-api-access-zm8fk\") pod \"maas-controller-787f676965-fs674\" (UID: \"e89e5261-15f6-4685-8240-79677d98527c\") " pod="opendatahub/maas-controller-787f676965-fs674" Jul 29 00:44:42.413243 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:42.413109 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e89e5261-15f6-4685-8240-79677d98527c-webhook-cert\") pod \"maas-controller-787f676965-fs674\" (UID: \"e89e5261-15f6-4685-8240-79677d98527c\") " pod="opendatahub/maas-controller-787f676965-fs674" Jul 29 00:44:42.514463 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:42.514419 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm8fk\" (UniqueName: \"kubernetes.io/projected/e89e5261-15f6-4685-8240-79677d98527c-kube-api-access-zm8fk\") pod \"maas-controller-787f676965-fs674\" (UID: \"e89e5261-15f6-4685-8240-79677d98527c\") " pod="opendatahub/maas-controller-787f676965-fs674" Jul 29 00:44:42.514463 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:42.514474 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e89e5261-15f6-4685-8240-79677d98527c-webhook-cert\") pod \"maas-controller-787f676965-fs674\" (UID: \"e89e5261-15f6-4685-8240-79677d98527c\") " pod="opendatahub/maas-controller-787f676965-fs674" Jul 29 00:44:42.516891 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:42.516869 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e89e5261-15f6-4685-8240-79677d98527c-webhook-cert\") pod \"maas-controller-787f676965-fs674\" (UID: \"e89e5261-15f6-4685-8240-79677d98527c\") " pod="opendatahub/maas-controller-787f676965-fs674" Jul 29 00:44:42.522295 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:42.522263 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm8fk\" (UniqueName: \"kubernetes.io/projected/e89e5261-15f6-4685-8240-79677d98527c-kube-api-access-zm8fk\") pod \"maas-controller-787f676965-fs674\" (UID: \"e89e5261-15f6-4685-8240-79677d98527c\") " pod="opendatahub/maas-controller-787f676965-fs674" Jul 29 00:44:42.646812 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:42.646734 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-787f676965-fs674" Jul 29 00:44:42.777746 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:42.777554 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-787f676965-fs674"] Jul 29 00:44:42.780521 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:44:42.780497 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode89e5261_15f6_4685_8240_79677d98527c.slice/crio-f211f2c739f3254e89943673fc408adb2bf4b571b69e02b073e03c2b636ed50c WatchSource:0}: Error finding container f211f2c739f3254e89943673fc408adb2bf4b571b69e02b073e03c2b636ed50c: Status 404 returned error can't find the container with id f211f2c739f3254e89943673fc408adb2bf4b571b69e02b073e03c2b636ed50c Jul 29 00:44:43.225396 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:43.225358 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-787f676965-fs674" event={"ID":"e89e5261-15f6-4685-8240-79677d98527c","Type":"ContainerStarted","Data":"f211f2c739f3254e89943673fc408adb2bf4b571b69e02b073e03c2b636ed50c"} Jul 29 00:44:45.235850 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:45.235815 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-787f676965-fs674" event={"ID":"e89e5261-15f6-4685-8240-79677d98527c","Type":"ContainerStarted","Data":"7a1729ffc02c9446329722224c95ad233f276644132dfdfc27a2221af4720380"} Jul 29 00:44:45.236290 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:45.235884 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-787f676965-fs674" Jul 29 00:44:45.277793 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:45.277701 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-787f676965-fs674" podStartSLOduration=1.03515419 podStartE2EDuration="3.277686576s" totalImagesPullingTime="2.242532386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:44:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:44:45.274281824 +0000 UTC m=+536.933507718" watchObservedRunningTime="2026-07-29 00:44:45.277686576 +0000 UTC m=+536.936912469" Jul 29 00:44:56.242163 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:44:56.242128 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-787f676965-fs674" Jul 29 00:45:18.413194 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.413158 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8"] Jul 29 00:45:18.420024 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.419996 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" Jul 29 00:45:18.422209 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.422184 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9jxwk\"" Jul 29 00:45:18.422358 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.422184 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 29 00:45:18.425810 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.425776 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8"] Jul 29 00:45:18.489392 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.489356 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpvmc\" (UniqueName: \"kubernetes.io/projected/02c5f00e-77b7-4f78-982f-655bcdd3e46d-kube-api-access-gpvmc\") pod \"maas-api-5f54d766bf-d4dr8\" (UID: \"02c5f00e-77b7-4f78-982f-655bcdd3e46d\") " pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" Jul 29 00:45:18.489576 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.489413 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c5f00e-77b7-4f78-982f-655bcdd3e46d-maas-api-tls\") pod \"maas-api-5f54d766bf-d4dr8\" (UID: \"02c5f00e-77b7-4f78-982f-655bcdd3e46d\") " pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" Jul 29 00:45:18.489576 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.489461 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c5f00e-77b7-4f78-982f-655bcdd3e46d-openshift-service-ca\") pod \"maas-api-5f54d766bf-d4dr8\" (UID: \"02c5f00e-77b7-4f78-982f-655bcdd3e46d\") " pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" Jul 29 00:45:18.590356 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.590299 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpvmc\" (UniqueName: \"kubernetes.io/projected/02c5f00e-77b7-4f78-982f-655bcdd3e46d-kube-api-access-gpvmc\") pod \"maas-api-5f54d766bf-d4dr8\" (UID: \"02c5f00e-77b7-4f78-982f-655bcdd3e46d\") " pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" Jul 29 00:45:18.590513 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.590458 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c5f00e-77b7-4f78-982f-655bcdd3e46d-maas-api-tls\") pod \"maas-api-5f54d766bf-d4dr8\" (UID: \"02c5f00e-77b7-4f78-982f-655bcdd3e46d\") " pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" Jul 29 00:45:18.590513 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.590511 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c5f00e-77b7-4f78-982f-655bcdd3e46d-openshift-service-ca\") pod \"maas-api-5f54d766bf-d4dr8\" (UID: \"02c5f00e-77b7-4f78-982f-655bcdd3e46d\") " pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" Jul 29 00:45:18.591192 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.591174 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02c5f00e-77b7-4f78-982f-655bcdd3e46d-openshift-service-ca\") pod \"maas-api-5f54d766bf-d4dr8\" (UID: \"02c5f00e-77b7-4f78-982f-655bcdd3e46d\") " pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" Jul 29 00:45:18.592781 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.592761 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02c5f00e-77b7-4f78-982f-655bcdd3e46d-maas-api-tls\") pod \"maas-api-5f54d766bf-d4dr8\" (UID: \"02c5f00e-77b7-4f78-982f-655bcdd3e46d\") " pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" Jul 29 00:45:18.597952 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.597927 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpvmc\" (UniqueName: \"kubernetes.io/projected/02c5f00e-77b7-4f78-982f-655bcdd3e46d-kube-api-access-gpvmc\") pod \"maas-api-5f54d766bf-d4dr8\" (UID: \"02c5f00e-77b7-4f78-982f-655bcdd3e46d\") " pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" Jul 29 00:45:18.731866 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.731790 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" Jul 29 00:45:18.858605 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.858571 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8"] Jul 29 00:45:18.863907 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.863881 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8" Jul 29 00:45:18.865671 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.865643 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 29 00:45:18.874070 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.874048 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8"] Jul 29 00:45:18.880209 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.880185 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8"] Jul 29 00:45:18.883272 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:45:18.883247 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02c5f00e_77b7_4f78_982f_655bcdd3e46d.slice/crio-385066bd79899ecde6e07a4467c3d7508f33e2da1b905490eaf988345bc472d6 WatchSource:0}: Error finding container 385066bd79899ecde6e07a4467c3d7508f33e2da1b905490eaf988345bc472d6: Status 404 returned error can't find the container with id 385066bd79899ecde6e07a4467c3d7508f33e2da1b905490eaf988345bc472d6 Jul 29 00:45:18.893826 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.893798 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz9l8\" (UniqueName: \"kubernetes.io/projected/6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9-kube-api-access-kz9l8\") pod \"payload-pre-processing-687dbb6cf6-tdjm8\" (UID: \"6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8" Jul 29 00:45:18.894085 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.894038 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-tdjm8\" (UID: \"6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8" Jul 29 00:45:18.995422 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.995330 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz9l8\" (UniqueName: \"kubernetes.io/projected/6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9-kube-api-access-kz9l8\") pod \"payload-pre-processing-687dbb6cf6-tdjm8\" (UID: \"6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8" Jul 29 00:45:18.995580 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.995469 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-tdjm8\" (UID: \"6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8" Jul 29 00:45:18.996184 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:18.996163 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-tdjm8\" (UID: \"6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8" Jul 29 00:45:19.002670 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:19.002648 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz9l8\" (UniqueName: \"kubernetes.io/projected/6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9-kube-api-access-kz9l8\") pod \"payload-pre-processing-687dbb6cf6-tdjm8\" (UID: \"6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8" Jul 29 00:45:19.177062 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:19.177024 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8" Jul 29 00:45:19.308403 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:19.308368 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8"] Jul 29 00:45:19.311774 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:45:19.311748 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b23a01d_67e9_4c9e_b88d_deecdf3bf1d9.slice/crio-3ee8dcf7b1745627d351f9d03743ce91a46ae1c7c2797704f253c2d4c958f357 WatchSource:0}: Error finding container 3ee8dcf7b1745627d351f9d03743ce91a46ae1c7c2797704f253c2d4c958f357: Status 404 returned error can't find the container with id 3ee8dcf7b1745627d351f9d03743ce91a46ae1c7c2797704f253c2d4c958f357 Jul 29 00:45:19.392204 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:19.392165 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" event={"ID":"02c5f00e-77b7-4f78-982f-655bcdd3e46d","Type":"ContainerStarted","Data":"385066bd79899ecde6e07a4467c3d7508f33e2da1b905490eaf988345bc472d6"} Jul 29 00:45:19.393532 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:19.393500 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8" event={"ID":"6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9","Type":"ContainerStarted","Data":"3ee8dcf7b1745627d351f9d03743ce91a46ae1c7c2797704f253c2d4c958f357"} Jul 29 00:45:22.417488 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:22.417453 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" event={"ID":"02c5f00e-77b7-4f78-982f-655bcdd3e46d","Type":"ContainerStarted","Data":"90176d4d0b95d7621394c03039c6b5ef65a8f8fa878622b58afe99e816897a44"} Jul 29 00:45:22.417979 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:22.417602 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" Jul 29 00:45:22.418803 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:22.418779 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8" event={"ID":"6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9","Type":"ContainerStarted","Data":"0f186e847735959acc4e71edd02c3afa85e18a37fdeb86abd436664afbb39362"} Jul 29 00:45:22.418906 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:22.418879 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8" Jul 29 00:45:22.434110 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:22.434067 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" podStartSLOduration=1.123298103 podStartE2EDuration="4.434053075s" totalImagesPullingTime="3.310754972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:45:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:45:22.432408841 +0000 UTC m=+574.091634733" watchObservedRunningTime="2026-07-29 00:45:22.434053075 +0000 UTC m=+574.093278970" Jul 29 00:45:26.438390 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:26.438355 2558 generic.go:356] "Generic (PLEG): container finished" podID="fb6b0154-70f0-474e-9bb5-b6f216183ba7" containerID="e3e25ad491dedff300a5b01858257804402f08d731e4c9782e642e29730db4d5" exitCode=0 Jul 29 00:45:26.438829 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:26.438415 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" event={"ID":"fb6b0154-70f0-474e-9bb5-b6f216183ba7","Type":"ContainerDied","Data":"e3e25ad491dedff300a5b01858257804402f08d731e4c9782e642e29730db4d5"} Jul 29 00:45:26.438896 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:26.438869 2558 scope.go:117] "RemoveContainer" containerID="e3e25ad491dedff300a5b01858257804402f08d731e4c9782e642e29730db4d5" Jul 29 00:45:26.439902 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:26.439870 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 00:45:26.439998 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:26.439954 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 00:45:26.440059 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:26.440002 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 00:45:26.457191 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:26.456910 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8" podStartSLOduration=5.550787517 podStartE2EDuration="8.456894146s" totalImagesPullingTime="2.906106629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:45:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:45:22.444998142 +0000 UTC m=+574.104224036" watchObservedRunningTime="2026-07-29 00:45:26.456894146 +0000 UTC m=+578.116120041" Jul 29 00:45:27.445566 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:27.445518 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" event={"ID":"fb6b0154-70f0-474e-9bb5-b6f216183ba7","Type":"ContainerStarted","Data":"16262a809bf3e42ba6c48eb2d7fef6ad8e5537d6c7ae95c6aaeecc0b5316232a"} Jul 29 00:45:27.940605 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:27.940574 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:45:27.941858 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:27.941831 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-l5dkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 29 00:45:27.941970 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:27.941877 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" podUID="fb6b0154-70f0-474e-9bb5-b6f216183ba7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 29 00:45:28.428549 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:28.428518 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5f54d766bf-d4dr8" Jul 29 00:45:28.940517 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:28.940486 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-l5dkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 29 00:45:28.940911 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:28.940549 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" podUID="fb6b0154-70f0-474e-9bb5-b6f216183ba7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 29 00:45:29.941365 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:29.941296 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-l5dkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 29 00:45:29.941804 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:29.941386 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" podUID="fb6b0154-70f0-474e-9bb5-b6f216183ba7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 29 00:45:30.940779 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:30.940740 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-l5dkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 29 00:45:30.940949 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:30.940796 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" podUID="fb6b0154-70f0-474e-9bb5-b6f216183ba7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 29 00:45:31.944440 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:31.944413 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:45:31.944832 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:31.944659 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:45:31.945445 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:31.945427 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-l5dkm" Jul 29 00:45:33.424462 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:33.424423 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8" podUID="6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.37:9004: connect: connection refused" Jul 29 00:45:43.424276 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:43.424242 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tdjm8" Jul 29 00:45:48.734461 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:48.734433 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 00:45:48.734869 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:48.734800 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 00:45:48.738930 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:48.738909 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 00:45:48.739056 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:45:48.738959 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 00:46:10.895925 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:10.895891 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz"] Jul 29 00:46:10.903339 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:10.903302 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:10.905676 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:10.905429 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 29 00:46:10.905676 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:10.905655 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 29 00:46:10.905918 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:10.905900 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7rgtn\"" Jul 29 00:46:10.906143 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:10.906126 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 29 00:46:10.907373 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:10.907349 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz"] Jul 29 00:46:10.941666 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:10.941634 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3994aaa8-24aa-4127-b4b2-73e0a604fac3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:10.941833 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:10.941672 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3994aaa8-24aa-4127-b4b2-73e0a604fac3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:10.941918 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:10.941827 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3994aaa8-24aa-4127-b4b2-73e0a604fac3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:10.941918 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:10.941878 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr99w\" (UniqueName: \"kubernetes.io/projected/3994aaa8-24aa-4127-b4b2-73e0a604fac3-kube-api-access-qr99w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:10.942015 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:10.941926 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3994aaa8-24aa-4127-b4b2-73e0a604fac3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:10.942015 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:10.941950 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3994aaa8-24aa-4127-b4b2-73e0a604fac3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:11.042464 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.042430 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3994aaa8-24aa-4127-b4b2-73e0a604fac3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:11.042464 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.042468 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3994aaa8-24aa-4127-b4b2-73e0a604fac3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:11.042690 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.042502 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3994aaa8-24aa-4127-b4b2-73e0a604fac3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:11.042690 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.042526 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3994aaa8-24aa-4127-b4b2-73e0a604fac3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:11.042690 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.042604 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3994aaa8-24aa-4127-b4b2-73e0a604fac3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:11.042690 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.042635 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr99w\" (UniqueName: \"kubernetes.io/projected/3994aaa8-24aa-4127-b4b2-73e0a604fac3-kube-api-access-qr99w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:11.042898 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.042876 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3994aaa8-24aa-4127-b4b2-73e0a604fac3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:11.042898 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.042890 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3994aaa8-24aa-4127-b4b2-73e0a604fac3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:11.043020 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.042991 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3994aaa8-24aa-4127-b4b2-73e0a604fac3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:11.044813 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.044786 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3994aaa8-24aa-4127-b4b2-73e0a604fac3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:11.045093 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.045072 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3994aaa8-24aa-4127-b4b2-73e0a604fac3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:11.053026 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.052998 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr99w\" (UniqueName: \"kubernetes.io/projected/3994aaa8-24aa-4127-b4b2-73e0a604fac3-kube-api-access-qr99w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hvlz\" (UID: \"3994aaa8-24aa-4127-b4b2-73e0a604fac3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:11.217041 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.216962 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:11.348059 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.348034 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz"] Jul 29 00:46:11.350633 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:46:11.350605 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3994aaa8_24aa_4127_b4b2_73e0a604fac3.slice/crio-6a2aab20605bc86bf44b8c117b25d237e81f065d9d5ac9c6cfeda0af0121d4b3 WatchSource:0}: Error finding container 6a2aab20605bc86bf44b8c117b25d237e81f065d9d5ac9c6cfeda0af0121d4b3: Status 404 returned error can't find the container with id 6a2aab20605bc86bf44b8c117b25d237e81f065d9d5ac9c6cfeda0af0121d4b3 Jul 29 00:46:11.351884 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.351864 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 00:46:11.658512 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:11.658422 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" event={"ID":"3994aaa8-24aa-4127-b4b2-73e0a604fac3","Type":"ContainerStarted","Data":"6a2aab20605bc86bf44b8c117b25d237e81f065d9d5ac9c6cfeda0af0121d4b3"} Jul 29 00:46:12.664188 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:12.664155 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" event={"ID":"3994aaa8-24aa-4127-b4b2-73e0a604fac3","Type":"ContainerStarted","Data":"70cdf214adb66ce63a2c0b6432f756f4c897404f3f8cb57e7b5ee4a6f16c943e"} Jul 29 00:46:12.681325 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:12.681252 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" podStartSLOduration=1.728252905 podStartE2EDuration="2.681235666s" totalImagesPullingTime="952.982761ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:46:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:46:12.67988825 +0000 UTC m=+624.339114144" watchObservedRunningTime="2026-07-29 00:46:12.681235666 +0000 UTC m=+624.340461559" Jul 29 00:46:21.217519 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:21.217464 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:21.217933 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:21.217551 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:21.229923 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:21.229897 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:21.715946 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:21.715920 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hvlz" Jul 29 00:46:50.721419 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.721390 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb"] Jul 29 00:46:50.725300 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.725279 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.727127 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.727105 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 29 00:46:50.738320 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.738282 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb"] Jul 29 00:46:50.769958 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.769928 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d224b23-8a91-416f-a013-1262fa2eaeeb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.770063 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.769972 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d224b23-8a91-416f-a013-1262fa2eaeeb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.770063 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.769993 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8sfg\" (UniqueName: \"kubernetes.io/projected/9d224b23-8a91-416f-a013-1262fa2eaeeb-kube-api-access-n8sfg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.770135 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.770071 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d224b23-8a91-416f-a013-1262fa2eaeeb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.770135 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.770099 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d224b23-8a91-416f-a013-1262fa2eaeeb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.770135 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.770127 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d224b23-8a91-416f-a013-1262fa2eaeeb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.870578 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.870541 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d224b23-8a91-416f-a013-1262fa2eaeeb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.870746 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.870594 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8sfg\" (UniqueName: \"kubernetes.io/projected/9d224b23-8a91-416f-a013-1262fa2eaeeb-kube-api-access-n8sfg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.870746 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.870669 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d224b23-8a91-416f-a013-1262fa2eaeeb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.870746 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.870702 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d224b23-8a91-416f-a013-1262fa2eaeeb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.870746 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.870734 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d224b23-8a91-416f-a013-1262fa2eaeeb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.870968 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.870800 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d224b23-8a91-416f-a013-1262fa2eaeeb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.871080 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.871060 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d224b23-8a91-416f-a013-1262fa2eaeeb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.871142 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.871117 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d224b23-8a91-416f-a013-1262fa2eaeeb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.871223 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.871207 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d224b23-8a91-416f-a013-1262fa2eaeeb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.873155 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.873128 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d224b23-8a91-416f-a013-1262fa2eaeeb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.873417 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.873397 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d224b23-8a91-416f-a013-1262fa2eaeeb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:50.877796 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:50.877765 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8sfg\" (UniqueName: \"kubernetes.io/projected/9d224b23-8a91-416f-a013-1262fa2eaeeb-kube-api-access-n8sfg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb\" (UID: \"9d224b23-8a91-416f-a013-1262fa2eaeeb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:51.038477 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:51.038387 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:46:51.185178 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:51.185071 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb"] Jul 29 00:46:51.188025 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:46:51.187999 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d224b23_8a91_416f_a013_1262fa2eaeeb.slice/crio-6f8ad1c8e7fb76f94ed9573bacfe75e6a3f65d5cda4738d174cc335bff707aa8 WatchSource:0}: Error finding container 6f8ad1c8e7fb76f94ed9573bacfe75e6a3f65d5cda4738d174cc335bff707aa8: Status 404 returned error can't find the container with id 6f8ad1c8e7fb76f94ed9573bacfe75e6a3f65d5cda4738d174cc335bff707aa8 Jul 29 00:46:51.845892 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:51.845854 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" event={"ID":"9d224b23-8a91-416f-a013-1262fa2eaeeb","Type":"ContainerStarted","Data":"6f8ad1c8e7fb76f94ed9573bacfe75e6a3f65d5cda4738d174cc335bff707aa8"} Jul 29 00:46:52.852986 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:52.852944 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" event={"ID":"9d224b23-8a91-416f-a013-1262fa2eaeeb","Type":"ContainerStarted","Data":"89a0aebcd4c710a3a94ae17579a2f11148659a0e928988f974d0be4d08a2b5b0"} Jul 29 00:46:52.874524 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:46:52.874474 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" podStartSLOduration=1.547579046 podStartE2EDuration="2.874437245s" totalImagesPullingTime="1.326858199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:46:52.87223691 +0000 UTC m=+664.531462992" watchObservedRunningTime="2026-07-29 00:46:52.874437245 +0000 UTC m=+664.533663138" Jul 29 00:47:01.038737 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:47:01.038697 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:47:01.039208 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:47:01.038805 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:47:01.051670 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:47:01.051640 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:47:01.912960 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:47:01.912920 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb" Jul 29 00:49:04.305923 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:04.305880 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-787f676965-fs674"] Jul 29 00:49:04.306478 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:04.306103 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-787f676965-fs674" podUID="e89e5261-15f6-4685-8240-79677d98527c" containerName="manager" containerID="cri-o://7a1729ffc02c9446329722224c95ad233f276644132dfdfc27a2221af4720380" gracePeriod=30 Jul 29 00:49:04.467756 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:04.467721 2558 generic.go:356] "Generic (PLEG): container finished" podID="e89e5261-15f6-4685-8240-79677d98527c" containerID="7a1729ffc02c9446329722224c95ad233f276644132dfdfc27a2221af4720380" exitCode=0 Jul 29 00:49:04.467921 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:04.467785 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-787f676965-fs674" event={"ID":"e89e5261-15f6-4685-8240-79677d98527c","Type":"ContainerDied","Data":"7a1729ffc02c9446329722224c95ad233f276644132dfdfc27a2221af4720380"} Jul 29 00:49:04.558542 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:04.558489 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-787f676965-fs674" Jul 29 00:49:04.572657 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:04.572637 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e89e5261-15f6-4685-8240-79677d98527c-webhook-cert\") pod \"e89e5261-15f6-4685-8240-79677d98527c\" (UID: \"e89e5261-15f6-4685-8240-79677d98527c\") " Jul 29 00:49:04.572739 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:04.572686 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zm8fk\" (UniqueName: \"kubernetes.io/projected/e89e5261-15f6-4685-8240-79677d98527c-kube-api-access-zm8fk\") pod \"e89e5261-15f6-4685-8240-79677d98527c\" (UID: \"e89e5261-15f6-4685-8240-79677d98527c\") " Jul 29 00:49:04.575687 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:04.575634 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e89e5261-15f6-4685-8240-79677d98527c-kube-api-access-zm8fk" (OuterVolumeSpecName: "kube-api-access-zm8fk") pod "e89e5261-15f6-4685-8240-79677d98527c" (UID: "e89e5261-15f6-4685-8240-79677d98527c"). InnerVolumeSpecName "kube-api-access-zm8fk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 00:49:04.575802 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:04.575767 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e89e5261-15f6-4685-8240-79677d98527c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e89e5261-15f6-4685-8240-79677d98527c" (UID: "e89e5261-15f6-4685-8240-79677d98527c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 00:49:04.674107 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:04.674080 2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e89e5261-15f6-4685-8240-79677d98527c-webhook-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:49:04.674107 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:04.674103 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zm8fk\" (UniqueName: \"kubernetes.io/projected/e89e5261-15f6-4685-8240-79677d98527c-kube-api-access-zm8fk\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:49:05.473783 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:05.473752 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-787f676965-fs674" Jul 29 00:49:05.473783 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:05.473762 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-787f676965-fs674" event={"ID":"e89e5261-15f6-4685-8240-79677d98527c","Type":"ContainerDied","Data":"f211f2c739f3254e89943673fc408adb2bf4b571b69e02b073e03c2b636ed50c"} Jul 29 00:49:05.474236 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:05.473820 2558 scope.go:117] "RemoveContainer" containerID="7a1729ffc02c9446329722224c95ad233f276644132dfdfc27a2221af4720380" Jul 29 00:49:05.486968 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:05.486944 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-787f676965-fs674"] Jul 29 00:49:05.490633 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:05.490611 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-787f676965-fs674"] Jul 29 00:49:06.789534 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:49:06.789503 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e89e5261-15f6-4685-8240-79677d98527c" path="/var/lib/kubelet/pods/e89e5261-15f6-4685-8240-79677d98527c/volumes" Jul 29 00:50:06.272600 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.272564 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-787f676965-zschv"] Jul 29 00:50:06.273127 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.272927 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e89e5261-15f6-4685-8240-79677d98527c" containerName="manager" Jul 29 00:50:06.273127 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.272937 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e89e5261-15f6-4685-8240-79677d98527c" containerName="manager" Jul 29 00:50:06.273127 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.272991 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e89e5261-15f6-4685-8240-79677d98527c" containerName="manager" Jul 29 00:50:06.277419 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.277397 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-787f676965-zschv" Jul 29 00:50:06.279798 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.279773 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 29 00:50:06.279918 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.279797 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 29 00:50:06.279918 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.279809 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xnhp8\"" Jul 29 00:50:06.284105 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.284049 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-787f676965-zschv"] Jul 29 00:50:06.416214 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.416183 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpr9w\" (UniqueName: \"kubernetes.io/projected/6220e0bd-752f-4156-8067-662a91238ea8-kube-api-access-zpr9w\") pod \"maas-controller-787f676965-zschv\" (UID: \"6220e0bd-752f-4156-8067-662a91238ea8\") " pod="opendatahub/maas-controller-787f676965-zschv" Jul 29 00:50:06.416394 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.416245 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6220e0bd-752f-4156-8067-662a91238ea8-webhook-cert\") pod \"maas-controller-787f676965-zschv\" (UID: \"6220e0bd-752f-4156-8067-662a91238ea8\") " pod="opendatahub/maas-controller-787f676965-zschv" Jul 29 00:50:06.516953 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.516904 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpr9w\" (UniqueName: \"kubernetes.io/projected/6220e0bd-752f-4156-8067-662a91238ea8-kube-api-access-zpr9w\") pod \"maas-controller-787f676965-zschv\" (UID: \"6220e0bd-752f-4156-8067-662a91238ea8\") " pod="opendatahub/maas-controller-787f676965-zschv" Jul 29 00:50:06.517167 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.517001 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6220e0bd-752f-4156-8067-662a91238ea8-webhook-cert\") pod \"maas-controller-787f676965-zschv\" (UID: \"6220e0bd-752f-4156-8067-662a91238ea8\") " pod="opendatahub/maas-controller-787f676965-zschv" Jul 29 00:50:06.519511 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.519492 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6220e0bd-752f-4156-8067-662a91238ea8-webhook-cert\") pod \"maas-controller-787f676965-zschv\" (UID: \"6220e0bd-752f-4156-8067-662a91238ea8\") " pod="opendatahub/maas-controller-787f676965-zschv" Jul 29 00:50:06.525386 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.525340 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpr9w\" (UniqueName: \"kubernetes.io/projected/6220e0bd-752f-4156-8067-662a91238ea8-kube-api-access-zpr9w\") pod \"maas-controller-787f676965-zschv\" (UID: \"6220e0bd-752f-4156-8067-662a91238ea8\") " pod="opendatahub/maas-controller-787f676965-zschv" Jul 29 00:50:06.589589 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.589563 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-787f676965-zschv" Jul 29 00:50:06.717541 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.717512 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-787f676965-zschv"] Jul 29 00:50:06.720276 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:50:06.720248 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6220e0bd_752f_4156_8067_662a91238ea8.slice/crio-c14ce8a990a0feb681197c8c2add0095e37da097e7789cd8313e472857ce1868 WatchSource:0}: Error finding container c14ce8a990a0feb681197c8c2add0095e37da097e7789cd8313e472857ce1868: Status 404 returned error can't find the container with id c14ce8a990a0feb681197c8c2add0095e37da097e7789cd8313e472857ce1868 Jul 29 00:50:06.761587 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:06.761557 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-787f676965-zschv" event={"ID":"6220e0bd-752f-4156-8067-662a91238ea8","Type":"ContainerStarted","Data":"c14ce8a990a0feb681197c8c2add0095e37da097e7789cd8313e472857ce1868"} Jul 29 00:50:07.767645 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:07.767607 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-787f676965-zschv" event={"ID":"6220e0bd-752f-4156-8067-662a91238ea8","Type":"ContainerStarted","Data":"f42cf461d06401f14cda59fe4b2271994a7d1cb243adc3f390b1b6d01eff79e3"} Jul 29 00:50:07.768005 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:07.767743 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-787f676965-zschv" Jul 29 00:50:07.784106 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:07.784064 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-787f676965-zschv" podStartSLOduration=1.252771535 podStartE2EDuration="1.784049362s" totalImagesPullingTime="531.277827ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:50:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:50:07.783112164 +0000 UTC m=+859.442338058" watchObservedRunningTime="2026-07-29 00:50:07.784049362 +0000 UTC m=+859.443275255" Jul 29 00:50:18.775108 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:18.775077 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-787f676965-zschv" Jul 29 00:50:23.434863 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.434833 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-787f676965-zschv"] Jul 29 00:50:23.435259 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.435017 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-787f676965-zschv" podUID="6220e0bd-752f-4156-8067-662a91238ea8" containerName="manager" containerID="cri-o://f42cf461d06401f14cda59fe4b2271994a7d1cb243adc3f390b1b6d01eff79e3" gracePeriod=30 Jul 29 00:50:23.687357 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.687287 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-787f676965-zschv" Jul 29 00:50:23.842501 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.842471 2558 generic.go:356] "Generic (PLEG): container finished" podID="6220e0bd-752f-4156-8067-662a91238ea8" containerID="f42cf461d06401f14cda59fe4b2271994a7d1cb243adc3f390b1b6d01eff79e3" exitCode=0 Jul 29 00:50:23.842666 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.842518 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-787f676965-zschv" Jul 29 00:50:23.842666 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.842561 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-787f676965-zschv" event={"ID":"6220e0bd-752f-4156-8067-662a91238ea8","Type":"ContainerDied","Data":"f42cf461d06401f14cda59fe4b2271994a7d1cb243adc3f390b1b6d01eff79e3"} Jul 29 00:50:23.842666 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.842615 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-787f676965-zschv" event={"ID":"6220e0bd-752f-4156-8067-662a91238ea8","Type":"ContainerDied","Data":"c14ce8a990a0feb681197c8c2add0095e37da097e7789cd8313e472857ce1868"} Jul 29 00:50:23.842666 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.842637 2558 scope.go:117] "RemoveContainer" containerID="f42cf461d06401f14cda59fe4b2271994a7d1cb243adc3f390b1b6d01eff79e3" Jul 29 00:50:23.851077 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.851060 2558 scope.go:117] "RemoveContainer" containerID="f42cf461d06401f14cda59fe4b2271994a7d1cb243adc3f390b1b6d01eff79e3" Jul 29 00:50:23.851330 ip-10-0-135-11 kubenswrapper[2558]: E0729 00:50:23.851295 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f42cf461d06401f14cda59fe4b2271994a7d1cb243adc3f390b1b6d01eff79e3\": container with ID starting with f42cf461d06401f14cda59fe4b2271994a7d1cb243adc3f390b1b6d01eff79e3 not found: ID does not exist" containerID="f42cf461d06401f14cda59fe4b2271994a7d1cb243adc3f390b1b6d01eff79e3" Jul 29 00:50:23.851438 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.851336 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f42cf461d06401f14cda59fe4b2271994a7d1cb243adc3f390b1b6d01eff79e3"} err="failed to get container status \"f42cf461d06401f14cda59fe4b2271994a7d1cb243adc3f390b1b6d01eff79e3\": rpc error: code = NotFound desc = could not find container \"f42cf461d06401f14cda59fe4b2271994a7d1cb243adc3f390b1b6d01eff79e3\": container with ID starting with f42cf461d06401f14cda59fe4b2271994a7d1cb243adc3f390b1b6d01eff79e3 not found: ID does not exist" Jul 29 00:50:23.862607 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.862589 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6220e0bd-752f-4156-8067-662a91238ea8-webhook-cert\") pod \"6220e0bd-752f-4156-8067-662a91238ea8\" (UID: \"6220e0bd-752f-4156-8067-662a91238ea8\") " Jul 29 00:50:23.862677 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.862616 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpr9w\" (UniqueName: \"kubernetes.io/projected/6220e0bd-752f-4156-8067-662a91238ea8-kube-api-access-zpr9w\") pod \"6220e0bd-752f-4156-8067-662a91238ea8\" (UID: \"6220e0bd-752f-4156-8067-662a91238ea8\") " Jul 29 00:50:23.864511 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.864484 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6220e0bd-752f-4156-8067-662a91238ea8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6220e0bd-752f-4156-8067-662a91238ea8" (UID: "6220e0bd-752f-4156-8067-662a91238ea8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 00:50:23.864598 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.864582 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6220e0bd-752f-4156-8067-662a91238ea8-kube-api-access-zpr9w" (OuterVolumeSpecName: "kube-api-access-zpr9w") pod "6220e0bd-752f-4156-8067-662a91238ea8" (UID: "6220e0bd-752f-4156-8067-662a91238ea8"). InnerVolumeSpecName "kube-api-access-zpr9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 00:50:23.964043 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.963957 2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6220e0bd-752f-4156-8067-662a91238ea8-webhook-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:50:23.964043 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:23.963988 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpr9w\" (UniqueName: \"kubernetes.io/projected/6220e0bd-752f-4156-8067-662a91238ea8-kube-api-access-zpr9w\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 00:50:24.163437 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:24.163410 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-787f676965-zschv"] Jul 29 00:50:24.166918 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:24.166894 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-787f676965-zschv"] Jul 29 00:50:24.788988 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:24.788954 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6220e0bd-752f-4156-8067-662a91238ea8" path="/var/lib/kubelet/pods/6220e0bd-752f-4156-8067-662a91238ea8/volumes" Jul 29 00:50:48.762882 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:48.762805 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 00:50:48.763413 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:48.763333 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 00:50:48.773732 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:48.773706 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 00:50:48.774020 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:50:48.773998 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 00:55:48.797483 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:55:48.797455 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 00:55:48.799971 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:55:48.798725 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 00:55:48.801891 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:55:48.801870 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 00:55:48.803277 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:55:48.803256 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 00:59:38.810626 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:38.810586 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2"] Jul 29 00:59:38.811147 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:38.811126 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6220e0bd-752f-4156-8067-662a91238ea8" containerName="manager" Jul 29 00:59:38.811218 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:38.811152 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6220e0bd-752f-4156-8067-662a91238ea8" containerName="manager" Jul 29 00:59:38.811282 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:38.811268 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6220e0bd-752f-4156-8067-662a91238ea8" containerName="manager" Jul 29 00:59:38.814139 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:38.814119 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:38.815835 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:38.815818 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-dc2cca86-kserve-self-signed-certs\"" Jul 29 00:59:38.822688 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:38.822665 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2"] Jul 29 00:59:38.945963 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:38.945924 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qdkk\" (UniqueName: \"kubernetes.io/projected/843821e5-8efa-43c9-81f5-514b6544941e-kube-api-access-6qdkk\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:38.945963 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:38.945966 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/843821e5-8efa-43c9-81f5-514b6544941e-tls-certs\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:38.946215 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:38.945987 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-model-cache\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:38.946215 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:38.946109 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-home\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:38.946215 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:38.946145 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-dshm\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:38.946215 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:38.946210 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-tmp-dir\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:39.046860 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.046828 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qdkk\" (UniqueName: \"kubernetes.io/projected/843821e5-8efa-43c9-81f5-514b6544941e-kube-api-access-6qdkk\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:39.046860 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.046863 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/843821e5-8efa-43c9-81f5-514b6544941e-tls-certs\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:39.047083 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.046893 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-model-cache\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:39.047083 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.046961 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-home\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:39.047083 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.046987 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-dshm\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:39.047083 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.047032 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-tmp-dir\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:39.047438 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.047401 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-model-cache\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:39.047520 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.047407 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-home\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:39.047520 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.047497 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-tmp-dir\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:39.049232 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.049211 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-dshm\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:39.049538 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.049520 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/843821e5-8efa-43c9-81f5-514b6544941e-tls-certs\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:39.054429 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.054406 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qdkk\" (UniqueName: \"kubernetes.io/projected/843821e5-8efa-43c9-81f5-514b6544941e-kube-api-access-6qdkk\") pod \"e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:39.123226 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.123196 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:39.259917 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.259892 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2"] Jul 29 00:59:39.262299 ip-10-0-135-11 kubenswrapper[2558]: W0729 00:59:39.262270 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod843821e5_8efa_43c9_81f5_514b6544941e.slice/crio-7102ce49e5bbe8338b44db97a03dc4acafe9390b3e2e04c46ad82a39f79471eb WatchSource:0}: Error finding container 7102ce49e5bbe8338b44db97a03dc4acafe9390b3e2e04c46ad82a39f79471eb: Status 404 returned error can't find the container with id 7102ce49e5bbe8338b44db97a03dc4acafe9390b3e2e04c46ad82a39f79471eb Jul 29 00:59:39.395986 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.395907 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" event={"ID":"843821e5-8efa-43c9-81f5-514b6544941e","Type":"ContainerStarted","Data":"535f9af4ef32fdcb845bcfd7f4507e4842a031dd7ffb4a6603ee47e54ed3c290"} Jul 29 00:59:39.395986 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.395945 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" event={"ID":"843821e5-8efa-43c9-81f5-514b6544941e","Type":"ContainerStarted","Data":"7102ce49e5bbe8338b44db97a03dc4acafe9390b3e2e04c46ad82a39f79471eb"} Jul 29 00:59:39.410998 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:39.410954 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" podStartSLOduration=1.410940643 podStartE2EDuration="1.410940643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:39.410006568 +0000 UTC m=+1431.069232452" watchObservedRunningTime="2026-07-29 00:59:39.410940643 +0000 UTC m=+1431.070166536" Jul 29 00:59:49.123429 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:49.123349 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:49.123429 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:49.123399 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:49.137284 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:49.137262 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 00:59:49.455245 ip-10-0-135-11 kubenswrapper[2558]: I0729 00:59:49.455163 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 01:00:16.886621 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:16.886587 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2"] Jul 29 01:00:31.905852 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:31.905809 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" podUID="843821e5-8efa-43c9-81f5-514b6544941e" containerName="main" containerID="cri-o://535f9af4ef32fdcb845bcfd7f4507e4842a031dd7ffb4a6603ee47e54ed3c290" gracePeriod=45 Jul 29 01:00:32.171250 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.171224 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 01:00:32.313897 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.313862 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qdkk\" (UniqueName: \"kubernetes.io/projected/843821e5-8efa-43c9-81f5-514b6544941e-kube-api-access-6qdkk\") pod \"843821e5-8efa-43c9-81f5-514b6544941e\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " Jul 29 01:00:32.314089 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.313905 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/843821e5-8efa-43c9-81f5-514b6544941e-tls-certs\") pod \"843821e5-8efa-43c9-81f5-514b6544941e\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " Jul 29 01:00:32.314089 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.313924 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-model-cache\") pod \"843821e5-8efa-43c9-81f5-514b6544941e\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " Jul 29 01:00:32.314089 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.313947 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-home\") pod \"843821e5-8efa-43c9-81f5-514b6544941e\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " Jul 29 01:00:32.314089 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.313996 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-dshm\") pod \"843821e5-8efa-43c9-81f5-514b6544941e\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " Jul 29 01:00:32.314089 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.314060 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-tmp-dir\") pod \"843821e5-8efa-43c9-81f5-514b6544941e\" (UID: \"843821e5-8efa-43c9-81f5-514b6544941e\") " Jul 29 01:00:32.314386 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.314161 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-model-cache" (OuterVolumeSpecName: "model-cache") pod "843821e5-8efa-43c9-81f5-514b6544941e" (UID: "843821e5-8efa-43c9-81f5-514b6544941e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:00:32.314386 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.314196 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-home" (OuterVolumeSpecName: "home") pod "843821e5-8efa-43c9-81f5-514b6544941e" (UID: "843821e5-8efa-43c9-81f5-514b6544941e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:00:32.314386 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.314378 2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-model-cache\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:00:32.314532 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.314398 2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-home\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:00:32.314532 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.314393 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "843821e5-8efa-43c9-81f5-514b6544941e" (UID: "843821e5-8efa-43c9-81f5-514b6544941e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:00:32.316262 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.316239 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-dshm" (OuterVolumeSpecName: "dshm") pod "843821e5-8efa-43c9-81f5-514b6544941e" (UID: "843821e5-8efa-43c9-81f5-514b6544941e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:00:32.316262 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.316258 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/843821e5-8efa-43c9-81f5-514b6544941e-kube-api-access-6qdkk" (OuterVolumeSpecName: "kube-api-access-6qdkk") pod "843821e5-8efa-43c9-81f5-514b6544941e" (UID: "843821e5-8efa-43c9-81f5-514b6544941e"). InnerVolumeSpecName "kube-api-access-6qdkk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:00:32.316441 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.316360 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/843821e5-8efa-43c9-81f5-514b6544941e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "843821e5-8efa-43c9-81f5-514b6544941e" (UID: "843821e5-8efa-43c9-81f5-514b6544941e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 01:00:32.415564 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.415486 2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-dshm\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:00:32.415564 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.415516 2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/843821e5-8efa-43c9-81f5-514b6544941e-tmp-dir\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:00:32.415564 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.415526 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qdkk\" (UniqueName: \"kubernetes.io/projected/843821e5-8efa-43c9-81f5-514b6544941e-kube-api-access-6qdkk\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:00:32.415564 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.415536 2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/843821e5-8efa-43c9-81f5-514b6544941e-tls-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:00:32.647599 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.647561 2558 generic.go:356] "Generic (PLEG): container finished" podID="843821e5-8efa-43c9-81f5-514b6544941e" containerID="535f9af4ef32fdcb845bcfd7f4507e4842a031dd7ffb4a6603ee47e54ed3c290" exitCode=0 Jul 29 01:00:32.647767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.647621 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" Jul 29 01:00:32.647767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.647645 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" event={"ID":"843821e5-8efa-43c9-81f5-514b6544941e","Type":"ContainerDied","Data":"535f9af4ef32fdcb845bcfd7f4507e4842a031dd7ffb4a6603ee47e54ed3c290"} Jul 29 01:00:32.647767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.647689 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2" event={"ID":"843821e5-8efa-43c9-81f5-514b6544941e","Type":"ContainerDied","Data":"7102ce49e5bbe8338b44db97a03dc4acafe9390b3e2e04c46ad82a39f79471eb"} Jul 29 01:00:32.647767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.647706 2558 scope.go:117] "RemoveContainer" containerID="535f9af4ef32fdcb845bcfd7f4507e4842a031dd7ffb4a6603ee47e54ed3c290" Jul 29 01:00:32.657016 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.656998 2558 scope.go:117] "RemoveContainer" containerID="535f9af4ef32fdcb845bcfd7f4507e4842a031dd7ffb4a6603ee47e54ed3c290" Jul 29 01:00:32.657288 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:00:32.657269 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"535f9af4ef32fdcb845bcfd7f4507e4842a031dd7ffb4a6603ee47e54ed3c290\": container with ID starting with 535f9af4ef32fdcb845bcfd7f4507e4842a031dd7ffb4a6603ee47e54ed3c290 not found: ID does not exist" containerID="535f9af4ef32fdcb845bcfd7f4507e4842a031dd7ffb4a6603ee47e54ed3c290" Jul 29 01:00:32.657362 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.657296 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"535f9af4ef32fdcb845bcfd7f4507e4842a031dd7ffb4a6603ee47e54ed3c290"} err="failed to get container status \"535f9af4ef32fdcb845bcfd7f4507e4842a031dd7ffb4a6603ee47e54ed3c290\": rpc error: code = NotFound desc = could not find container \"535f9af4ef32fdcb845bcfd7f4507e4842a031dd7ffb4a6603ee47e54ed3c290\": container with ID starting with 535f9af4ef32fdcb845bcfd7f4507e4842a031dd7ffb4a6603ee47e54ed3c290 not found: ID does not exist" Jul 29 01:00:32.669366 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.669303 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2"] Jul 29 01:00:32.673250 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.673230 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-dc2cca86-kserve-66f79fb97f-blbp2"] Jul 29 01:00:32.790169 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:32.790144 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="843821e5-8efa-43c9-81f5-514b6544941e" path="/var/lib/kubelet/pods/843821e5-8efa-43c9-81f5-514b6544941e/volumes" Jul 29 01:00:48.826049 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:48.826023 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:00:48.828503 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:48.827909 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:00:48.830607 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:48.830587 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 01:00:48.832581 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:00:48.832558 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 01:03:56.843266 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.843232 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9"] Jul 29 01:03:56.843858 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.843607 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="843821e5-8efa-43c9-81f5-514b6544941e" containerName="main" Jul 29 01:03:56.843858 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.843621 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="843821e5-8efa-43c9-81f5-514b6544941e" containerName="main" Jul 29 01:03:56.843858 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.843702 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="843821e5-8efa-43c9-81f5-514b6544941e" containerName="main" Jul 29 01:03:56.846171 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.846153 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:56.848143 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.848122 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f6bbaf0d-openshift-default-dockercfg-c9f8r\"" Jul 29 01:03:56.866983 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.866962 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9"] Jul 29 01:03:56.934331 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.934276 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/487904a4-c22c-4544-9506-f4582b02b144-istio-token\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:56.934468 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.934339 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/487904a4-c22c-4544-9506-f4582b02b144-istio-podinfo\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:56.934468 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.934374 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-workload-certs\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:56.934468 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.934417 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-istio-envoy\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:56.934634 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.934497 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc2qh\" (UniqueName: \"kubernetes.io/projected/487904a4-c22c-4544-9506-f4582b02b144-kube-api-access-hc2qh\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:56.934634 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.934550 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-istio-data\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:56.934634 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.934598 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-credential-socket\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:56.934778 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.934657 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/487904a4-c22c-4544-9506-f4582b02b144-istiod-ca-cert\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:56.934778 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:56.934708 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-workload-socket\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.035221 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.035189 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/487904a4-c22c-4544-9506-f4582b02b144-istiod-ca-cert\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.035221 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.035222 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-workload-socket\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.035469 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.035243 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/487904a4-c22c-4544-9506-f4582b02b144-istio-token\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.035469 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.035270 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/487904a4-c22c-4544-9506-f4582b02b144-istio-podinfo\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.035469 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.035298 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-workload-certs\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.035469 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.035363 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-istio-envoy\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.035681 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.035487 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc2qh\" (UniqueName: \"kubernetes.io/projected/487904a4-c22c-4544-9506-f4582b02b144-kube-api-access-hc2qh\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.035681 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.035530 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-istio-data\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.035681 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.035580 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-credential-socket\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.035840 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.035754 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-workload-certs\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.035903 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.035850 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-istio-data\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.035903 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.035866 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-workload-socket\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.036003 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.035904 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-credential-socket\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.036061 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.036014 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/487904a4-c22c-4544-9506-f4582b02b144-istiod-ca-cert\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.037558 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.037539 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-istio-envoy\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.037816 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.037798 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/487904a4-c22c-4544-9506-f4582b02b144-istio-podinfo\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.042767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.042747 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/487904a4-c22c-4544-9506-f4582b02b144-istio-token\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.043116 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.043088 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc2qh\" (UniqueName: \"kubernetes.io/projected/487904a4-c22c-4544-9506-f4582b02b144-kube-api-access-hc2qh\") pod \"e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.158075 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.158005 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:57.296574 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.296540 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9"] Jul 29 01:03:57.300640 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:03:57.300611 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod487904a4_c22c_4544_9506_f4582b02b144.slice/crio-06943b0a38153b264c4071869d67d0635ae8bad7b1c8d5d90bb5b64c749aec69 WatchSource:0}: Error finding container 06943b0a38153b264c4071869d67d0635ae8bad7b1c8d5d90bb5b64c749aec69: Status 404 returned error can't find the container with id 06943b0a38153b264c4071869d67d0635ae8bad7b1c8d5d90bb5b64c749aec69 Jul 29 01:03:57.302718 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.302679 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:03:57.302780 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.302743 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:03:57.302780 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.302771 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:03:57.612256 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.612214 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" event={"ID":"487904a4-c22c-4544-9506-f4582b02b144","Type":"ContainerStarted","Data":"f0b2d725a5e48b03fe05455bc53a3c93560fd2996c5161885daa17c823668d2e"} Jul 29 01:03:57.612450 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:57.612277 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" event={"ID":"487904a4-c22c-4544-9506-f4582b02b144","Type":"ContainerStarted","Data":"06943b0a38153b264c4071869d67d0635ae8bad7b1c8d5d90bb5b64c749aec69"} Jul 29 01:03:58.158634 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:58.158599 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:58.163590 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:58.163558 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:58.183114 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:58.183062 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" podStartSLOduration=2.183047326 podStartE2EDuration="2.183047326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:03:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:03:57.628300328 +0000 UTC m=+1689.287526257" watchObservedRunningTime="2026-07-29 01:03:58.183047326 +0000 UTC m=+1689.842273218" Jul 29 01:03:58.616643 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:58.616616 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:58.617630 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:58.617611 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:03:59.149754 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.149723 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc"] Jul 29 01:03:59.153276 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.153254 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:03:59.154917 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.154897 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f6bbaf0d-serving-cert\"" Jul 29 01:03:59.169134 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.169110 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc"] Jul 29 01:03:59.254217 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.254178 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcffh\" (UniqueName: \"kubernetes.io/projected/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-kube-api-access-xcffh\") pod \"maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc\" (UID: \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:03:59.254416 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.254247 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-maas-api-tls\") pod \"maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc\" (UID: \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:03:59.254416 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.254301 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-openshift-service-ca\") pod \"maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc\" (UID: \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:03:59.355695 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.355662 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcffh\" (UniqueName: \"kubernetes.io/projected/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-kube-api-access-xcffh\") pod \"maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc\" (UID: \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:03:59.355873 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.355712 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-maas-api-tls\") pod \"maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc\" (UID: \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:03:59.355939 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.355903 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-openshift-service-ca\") pod \"maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc\" (UID: \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:03:59.356559 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.356536 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-openshift-service-ca\") pod \"maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc\" (UID: \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:03:59.358038 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.358019 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-maas-api-tls\") pod \"maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc\" (UID: \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:03:59.363015 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.362993 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcffh\" (UniqueName: \"kubernetes.io/projected/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-kube-api-access-xcffh\") pod \"maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc\" (UID: \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:03:59.464515 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.464426 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:03:59.609036 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.609003 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc"] Jul 29 01:03:59.624188 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:03:59.624161 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17fc12ba_26d2_4814_a7c4_0fcf4abf4755.slice/crio-110ea965714591187776f734406f9b5c2cb0666ecbabe1baf8c2048ff569b076 WatchSource:0}: Error finding container 110ea965714591187776f734406f9b5c2cb0666ecbabe1baf8c2048ff569b076: Status 404 returned error can't find the container with id 110ea965714591187776f734406f9b5c2cb0666ecbabe1baf8c2048ff569b076 Jul 29 01:03:59.625492 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:03:59.625474 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 01:04:00.628672 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:00.628636 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" event={"ID":"17fc12ba-26d2-4814-a7c4-0fcf4abf4755","Type":"ContainerStarted","Data":"110ea965714591187776f734406f9b5c2cb0666ecbabe1baf8c2048ff569b076"} Jul 29 01:04:01.635461 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:01.635428 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" event={"ID":"17fc12ba-26d2-4814-a7c4-0fcf4abf4755","Type":"ContainerStarted","Data":"3f322734237145561fda084ecab603b1d4fcc8091fa16b6903ddfabbb137217d"} Jul 29 01:04:01.635897 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:01.635552 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:04:01.652163 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:01.652117 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" podStartSLOduration=1.592088064 podStartE2EDuration="2.652100558s" totalImagesPullingTime="1.060012494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:04:01.650885218 +0000 UTC m=+1693.310111103" watchObservedRunningTime="2026-07-29 01:04:01.652100558 +0000 UTC m=+1693.311326451" Jul 29 01:04:07.646627 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:07.646595 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:04:14.657468 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:14.657424 2558 patch_prober.go:28] interesting pod/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 29 01:04:14.657862 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:14.657488 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" podUID="487904a4-c22c-4544-9506-f4582b02b144" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 29 01:04:15.710473 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:15.710438 2558 generic.go:356] "Generic (PLEG): container finished" podID="487904a4-c22c-4544-9506-f4582b02b144" containerID="f0b2d725a5e48b03fe05455bc53a3c93560fd2996c5161885daa17c823668d2e" exitCode=0 Jul 29 01:04:15.710950 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:15.710479 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" event={"ID":"487904a4-c22c-4544-9506-f4582b02b144","Type":"ContainerDied","Data":"f0b2d725a5e48b03fe05455bc53a3c93560fd2996c5161885daa17c823668d2e"} Jul 29 01:04:15.710950 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:15.710835 2558 scope.go:117] "RemoveContainer" containerID="f0b2d725a5e48b03fe05455bc53a3c93560fd2996c5161885daa17c823668d2e" Jul 29 01:04:15.711859 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:15.711827 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:04:15.711917 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:15.711900 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:04:15.711952 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:15.711931 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:04:16.717073 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:16.717041 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" event={"ID":"487904a4-c22c-4544-9506-f4582b02b144","Type":"ContainerStarted","Data":"5f789dfdb292cb73d2f2c409fd41837d0ad032ba612f2017263fedc2a9d71b44"} Jul 29 01:04:17.158961 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:17.158911 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:04:17.159186 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:17.159091 2558 patch_prober.go:28] interesting pod/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 29 01:04:17.159186 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:17.159150 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" podUID="487904a4-c22c-4544-9506-f4582b02b144" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 29 01:04:18.159807 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:18.159772 2558 patch_prober.go:28] interesting pod/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 29 01:04:18.160404 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:18.160364 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" podUID="487904a4-c22c-4544-9506-f4582b02b144" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 29 01:04:19.158527 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:19.158489 2558 patch_prober.go:28] interesting pod/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 29 01:04:19.158810 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:19.158781 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" podUID="487904a4-c22c-4544-9506-f4582b02b144" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 29 01:04:20.159147 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:20.159108 2558 patch_prober.go:28] interesting pod/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 29 01:04:20.159738 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:20.159708 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" podUID="487904a4-c22c-4544-9506-f4582b02b144" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 29 01:04:21.161862 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:21.161832 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:04:21.162229 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:21.162156 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:04:21.162839 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:21.162821 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:04:29.530275 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.530238 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc"] Jul 29 01:04:29.530731 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.530538 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" podUID="17fc12ba-26d2-4814-a7c4-0fcf4abf4755" containerName="maas-api" containerID="cri-o://3f322734237145561fda084ecab603b1d4fcc8091fa16b6903ddfabbb137217d" gracePeriod=30 Jul 29 01:04:29.782695 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.782630 2558 generic.go:356] "Generic (PLEG): container finished" podID="17fc12ba-26d2-4814-a7c4-0fcf4abf4755" containerID="3f322734237145561fda084ecab603b1d4fcc8091fa16b6903ddfabbb137217d" exitCode=0 Jul 29 01:04:29.782806 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.782704 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" event={"ID":"17fc12ba-26d2-4814-a7c4-0fcf4abf4755","Type":"ContainerDied","Data":"3f322734237145561fda084ecab603b1d4fcc8091fa16b6903ddfabbb137217d"} Jul 29 01:04:29.782806 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.782758 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" event={"ID":"17fc12ba-26d2-4814-a7c4-0fcf4abf4755","Type":"ContainerDied","Data":"110ea965714591187776f734406f9b5c2cb0666ecbabe1baf8c2048ff569b076"} Jul 29 01:04:29.782806 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.782776 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="110ea965714591187776f734406f9b5c2cb0666ecbabe1baf8c2048ff569b076" Jul 29 01:04:29.786124 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.786102 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:04:29.841967 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.841584 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-maas-api-tls\") pod \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\" (UID: \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\") " Jul 29 01:04:29.842140 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.842091 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcffh\" (UniqueName: \"kubernetes.io/projected/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-kube-api-access-xcffh\") pod \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\" (UID: \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\") " Jul 29 01:04:29.842140 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.842126 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-openshift-service-ca\") pod \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\" (UID: \"17fc12ba-26d2-4814-a7c4-0fcf4abf4755\") " Jul 29 01:04:29.842798 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.842762 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "17fc12ba-26d2-4814-a7c4-0fcf4abf4755" (UID: "17fc12ba-26d2-4814-a7c4-0fcf4abf4755"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:04:29.844633 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.844604 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9"] Jul 29 01:04:29.844906 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.844811 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" podUID="487904a4-c22c-4544-9506-f4582b02b144" containerName="istio-proxy" containerID="cri-o://5f789dfdb292cb73d2f2c409fd41837d0ad032ba612f2017263fedc2a9d71b44" gracePeriod=30 Jul 29 01:04:29.845330 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.845285 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-kube-api-access-xcffh" (OuterVolumeSpecName: "kube-api-access-xcffh") pod "17fc12ba-26d2-4814-a7c4-0fcf4abf4755" (UID: "17fc12ba-26d2-4814-a7c4-0fcf4abf4755"). InnerVolumeSpecName "kube-api-access-xcffh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:04:29.845330 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.845290 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "17fc12ba-26d2-4814-a7c4-0fcf4abf4755" (UID: "17fc12ba-26d2-4814-a7c4-0fcf4abf4755"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 01:04:29.943702 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.943669 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcffh\" (UniqueName: \"kubernetes.io/projected/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-kube-api-access-xcffh\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:04:29.943702 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.943696 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-openshift-service-ca\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:04:29.943878 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:29.943713 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17fc12ba-26d2-4814-a7c4-0fcf4abf4755-maas-api-tls\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:04:30.788844 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.788812 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc" Jul 29 01:04:30.813179 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.813151 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc"] Jul 29 01:04:30.821902 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.821877 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f6bbaf0d-558f665f45-pfggc"] Jul 29 01:04:30.829336 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.829282 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62"] Jul 29 01:04:30.829712 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.829695 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17fc12ba-26d2-4814-a7c4-0fcf4abf4755" containerName="maas-api" Jul 29 01:04:30.829784 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.829716 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="17fc12ba-26d2-4814-a7c4-0fcf4abf4755" containerName="maas-api" Jul 29 01:04:30.829840 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.829819 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="17fc12ba-26d2-4814-a7c4-0fcf4abf4755" containerName="maas-api" Jul 29 01:04:30.832192 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.832173 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:30.833861 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.833841 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-956869dd-gw-openshift-default-dockercfg-qq7gj\"" Jul 29 01:04:30.842893 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.842868 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62"] Jul 29 01:04:30.951730 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.951687 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnsv2\" (UniqueName: \"kubernetes.io/projected/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-kube-api-access-pnsv2\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:30.951905 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.951735 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-token\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:30.951905 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.951832 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istiod-ca-cert\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:30.951905 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.951886 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-workload-socket\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:30.952035 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.951959 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-credential-socket\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:30.952035 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.951983 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-workload-certs\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:30.952035 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.952012 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-data\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:30.952155 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.952075 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-envoy\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:30.952155 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:30.952103 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-podinfo\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.052824 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.052746 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnsv2\" (UniqueName: \"kubernetes.io/projected/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-kube-api-access-pnsv2\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.052824 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.052782 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-token\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.053008 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.052836 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istiod-ca-cert\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.053008 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.052876 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-workload-socket\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.053008 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.052936 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-credential-socket\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.053008 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.052959 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-workload-certs\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.053008 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.052992 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-data\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.053275 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.053037 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-envoy\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.053275 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.053119 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-podinfo\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.053402 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.053361 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-credential-socket\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.053532 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.053503 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-workload-certs\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.053689 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.053544 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-data\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.053689 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.053563 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-workload-socket\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.053831 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.053810 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istiod-ca-cert\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.055385 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.055366 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-envoy\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.055681 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.055661 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-podinfo\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.059816 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.059797 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-token\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.060187 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.060167 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnsv2\" (UniqueName: \"kubernetes.io/projected/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-kube-api-access-pnsv2\") pod \"e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.143501 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.143459 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:31.279299 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.279206 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62"] Jul 29 01:04:31.281950 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:04:31.281921 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c5762a7_aaeb_48b3_9028_2f24526cf2e8.slice/crio-b6f1ab65559a0b9f6c3b458e768402b30b994c98a786dd68816e60225deaeee6 WatchSource:0}: Error finding container b6f1ab65559a0b9f6c3b458e768402b30b994c98a786dd68816e60225deaeee6: Status 404 returned error can't find the container with id b6f1ab65559a0b9f6c3b458e768402b30b994c98a786dd68816e60225deaeee6 Jul 29 01:04:31.284330 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.284284 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:04:31.284436 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.284373 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:04:31.284436 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.284425 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:04:31.795409 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.795304 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" event={"ID":"6c5762a7-aaeb-48b3-9028-2f24526cf2e8","Type":"ContainerStarted","Data":"a0f28135e460a0b5eef642123e3ca73c6b2fb4bcdf900b9056cf86ee32a19dce"} Jul 29 01:04:31.795409 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.795371 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" event={"ID":"6c5762a7-aaeb-48b3-9028-2f24526cf2e8","Type":"ContainerStarted","Data":"b6f1ab65559a0b9f6c3b458e768402b30b994c98a786dd68816e60225deaeee6"} Jul 29 01:04:31.814761 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:31.814715 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" podStartSLOduration=1.814700491 podStartE2EDuration="1.814700491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:04:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:04:31.81267335 +0000 UTC m=+1723.471899243" watchObservedRunningTime="2026-07-29 01:04:31.814700491 +0000 UTC m=+1723.473926384" Jul 29 01:04:32.144480 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:32.144441 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:32.149698 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:32.149676 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:32.789850 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:32.789813 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17fc12ba-26d2-4814-a7c4-0fcf4abf4755" path="/var/lib/kubelet/pods/17fc12ba-26d2-4814-a7c4-0fcf4abf4755/volumes" Jul 29 01:04:32.799711 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:32.799691 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:32.800471 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:32.800453 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:35.091678 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.091652 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:04:35.192563 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.192472 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc2qh\" (UniqueName: \"kubernetes.io/projected/487904a4-c22c-4544-9506-f4582b02b144-kube-api-access-hc2qh\") pod \"487904a4-c22c-4544-9506-f4582b02b144\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " Jul 29 01:04:35.192563 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.192518 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/487904a4-c22c-4544-9506-f4582b02b144-istiod-ca-cert\") pod \"487904a4-c22c-4544-9506-f4582b02b144\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " Jul 29 01:04:35.192767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.192592 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-istio-envoy\") pod \"487904a4-c22c-4544-9506-f4582b02b144\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " Jul 29 01:04:35.192767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.192615 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-workload-socket\") pod \"487904a4-c22c-4544-9506-f4582b02b144\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " Jul 29 01:04:35.192767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.192633 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-istio-data\") pod \"487904a4-c22c-4544-9506-f4582b02b144\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " Jul 29 01:04:35.192767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.192672 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-credential-socket\") pod \"487904a4-c22c-4544-9506-f4582b02b144\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " Jul 29 01:04:35.192767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.192726 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/487904a4-c22c-4544-9506-f4582b02b144-istio-podinfo\") pod \"487904a4-c22c-4544-9506-f4582b02b144\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " Jul 29 01:04:35.192767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.192761 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/487904a4-c22c-4544-9506-f4582b02b144-istio-token\") pod \"487904a4-c22c-4544-9506-f4582b02b144\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " Jul 29 01:04:35.193052 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.192824 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-workload-certs\") pod \"487904a4-c22c-4544-9506-f4582b02b144\" (UID: \"487904a4-c22c-4544-9506-f4582b02b144\") " Jul 29 01:04:35.193052 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.192915 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "487904a4-c22c-4544-9506-f4582b02b144" (UID: "487904a4-c22c-4544-9506-f4582b02b144"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:04:35.193052 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.192934 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/487904a4-c22c-4544-9506-f4582b02b144-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "487904a4-c22c-4544-9506-f4582b02b144" (UID: "487904a4-c22c-4544-9506-f4582b02b144"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:04:35.193187 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.193143 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "487904a4-c22c-4544-9506-f4582b02b144" (UID: "487904a4-c22c-4544-9506-f4582b02b144"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:04:35.193240 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.193182 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "487904a4-c22c-4544-9506-f4582b02b144" (UID: "487904a4-c22c-4544-9506-f4582b02b144"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:04:35.193345 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.193268 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/487904a4-c22c-4544-9506-f4582b02b144-istiod-ca-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:04:35.193345 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.193289 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-workload-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:04:35.193907 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.193865 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-istio-data" (OuterVolumeSpecName: "istio-data") pod "487904a4-c22c-4544-9506-f4582b02b144" (UID: "487904a4-c22c-4544-9506-f4582b02b144"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:04:35.195228 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.195199 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/487904a4-c22c-4544-9506-f4582b02b144-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "487904a4-c22c-4544-9506-f4582b02b144" (UID: "487904a4-c22c-4544-9506-f4582b02b144"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 01:04:35.195374 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.195270 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/487904a4-c22c-4544-9506-f4582b02b144-kube-api-access-hc2qh" (OuterVolumeSpecName: "kube-api-access-hc2qh") pod "487904a4-c22c-4544-9506-f4582b02b144" (UID: "487904a4-c22c-4544-9506-f4582b02b144"). InnerVolumeSpecName "kube-api-access-hc2qh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:04:35.195429 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.195405 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "487904a4-c22c-4544-9506-f4582b02b144" (UID: "487904a4-c22c-4544-9506-f4582b02b144"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:04:35.195612 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.195598 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/487904a4-c22c-4544-9506-f4582b02b144-istio-token" (OuterVolumeSpecName: "istio-token") pod "487904a4-c22c-4544-9506-f4582b02b144" (UID: "487904a4-c22c-4544-9506-f4582b02b144"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:04:35.294708 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.294673 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-workload-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:04:35.294708 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.294706 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc2qh\" (UniqueName: \"kubernetes.io/projected/487904a4-c22c-4544-9506-f4582b02b144-kube-api-access-hc2qh\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:04:35.294708 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.294718 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-istio-envoy\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:04:35.294887 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.294726 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-istio-data\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:04:35.294887 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.294735 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/487904a4-c22c-4544-9506-f4582b02b144-credential-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:04:35.294887 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.294742 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/487904a4-c22c-4544-9506-f4582b02b144-istio-podinfo\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:04:35.294887 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.294751 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/487904a4-c22c-4544-9506-f4582b02b144-istio-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:04:35.815375 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.815345 2558 generic.go:356] "Generic (PLEG): container finished" podID="487904a4-c22c-4544-9506-f4582b02b144" containerID="5f789dfdb292cb73d2f2c409fd41837d0ad032ba612f2017263fedc2a9d71b44" exitCode=0 Jul 29 01:04:35.815543 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.815384 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" Jul 29 01:04:35.815543 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.815391 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" event={"ID":"487904a4-c22c-4544-9506-f4582b02b144","Type":"ContainerDied","Data":"5f789dfdb292cb73d2f2c409fd41837d0ad032ba612f2017263fedc2a9d71b44"} Jul 29 01:04:35.815543 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.815426 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9" event={"ID":"487904a4-c22c-4544-9506-f4582b02b144","Type":"ContainerDied","Data":"06943b0a38153b264c4071869d67d0635ae8bad7b1c8d5d90bb5b64c749aec69"} Jul 29 01:04:35.815543 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.815447 2558 scope.go:117] "RemoveContainer" containerID="5f789dfdb292cb73d2f2c409fd41837d0ad032ba612f2017263fedc2a9d71b44" Jul 29 01:04:35.824696 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.824679 2558 scope.go:117] "RemoveContainer" containerID="f0b2d725a5e48b03fe05455bc53a3c93560fd2996c5161885daa17c823668d2e" Jul 29 01:04:35.832775 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.832749 2558 scope.go:117] "RemoveContainer" containerID="5f789dfdb292cb73d2f2c409fd41837d0ad032ba612f2017263fedc2a9d71b44" Jul 29 01:04:35.833180 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:04:35.833123 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f789dfdb292cb73d2f2c409fd41837d0ad032ba612f2017263fedc2a9d71b44\": container with ID starting with 5f789dfdb292cb73d2f2c409fd41837d0ad032ba612f2017263fedc2a9d71b44 not found: ID does not exist" containerID="5f789dfdb292cb73d2f2c409fd41837d0ad032ba612f2017263fedc2a9d71b44" Jul 29 01:04:35.833275 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.833191 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f789dfdb292cb73d2f2c409fd41837d0ad032ba612f2017263fedc2a9d71b44"} err="failed to get container status \"5f789dfdb292cb73d2f2c409fd41837d0ad032ba612f2017263fedc2a9d71b44\": rpc error: code = NotFound desc = could not find container \"5f789dfdb292cb73d2f2c409fd41837d0ad032ba612f2017263fedc2a9d71b44\": container with ID starting with 5f789dfdb292cb73d2f2c409fd41837d0ad032ba612f2017263fedc2a9d71b44 not found: ID does not exist" Jul 29 01:04:35.833275 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.833223 2558 scope.go:117] "RemoveContainer" containerID="f0b2d725a5e48b03fe05455bc53a3c93560fd2996c5161885daa17c823668d2e" Jul 29 01:04:35.833706 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:04:35.833550 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0b2d725a5e48b03fe05455bc53a3c93560fd2996c5161885daa17c823668d2e\": container with ID starting with f0b2d725a5e48b03fe05455bc53a3c93560fd2996c5161885daa17c823668d2e not found: ID does not exist" containerID="f0b2d725a5e48b03fe05455bc53a3c93560fd2996c5161885daa17c823668d2e" Jul 29 01:04:35.833706 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.833581 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0b2d725a5e48b03fe05455bc53a3c93560fd2996c5161885daa17c823668d2e"} err="failed to get container status \"f0b2d725a5e48b03fe05455bc53a3c93560fd2996c5161885daa17c823668d2e\": rpc error: code = NotFound desc = could not find container \"f0b2d725a5e48b03fe05455bc53a3c93560fd2996c5161885daa17c823668d2e\": container with ID starting with f0b2d725a5e48b03fe05455bc53a3c93560fd2996c5161885daa17c823668d2e not found: ID does not exist" Jul 29 01:04:35.835612 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.835594 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9"] Jul 29 01:04:35.838799 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:35.838782 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f6bbaf0d-openshift-default-57bccd78db-spcd9"] Jul 29 01:04:36.789715 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:36.789677 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="487904a4-c22c-4544-9506-f4582b02b144" path="/var/lib/kubelet/pods/487904a4-c22c-4544-9506-f4582b02b144/volumes" Jul 29 01:04:47.800937 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:47.800858 2558 patch_prober.go:28] interesting pod/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 29 01:04:47.800937 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:47.800922 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 29 01:04:47.876016 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:47.875983 2558 generic.go:356] "Generic (PLEG): container finished" podID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerID="a0f28135e460a0b5eef642123e3ca73c6b2fb4bcdf900b9056cf86ee32a19dce" exitCode=0 Jul 29 01:04:47.876176 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:47.876059 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" event={"ID":"6c5762a7-aaeb-48b3-9028-2f24526cf2e8","Type":"ContainerDied","Data":"a0f28135e460a0b5eef642123e3ca73c6b2fb4bcdf900b9056cf86ee32a19dce"} Jul 29 01:04:47.876426 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:47.876412 2558 scope.go:117] "RemoveContainer" containerID="a0f28135e460a0b5eef642123e3ca73c6b2fb4bcdf900b9056cf86ee32a19dce" Jul 29 01:04:47.877460 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:47.877430 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:04:47.877534 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:47.877516 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:04:47.877591 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:47.877567 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:04:48.883246 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:48.883206 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" event={"ID":"6c5762a7-aaeb-48b3-9028-2f24526cf2e8","Type":"ContainerStarted","Data":"64809ce223785a4c999acd96579949406c38a16000992160e80ca40af69d1e75"} Jul 29 01:04:49.144435 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:49.144349 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:49.144678 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:49.144653 2558 patch_prober.go:28] interesting pod/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 29 01:04:49.144763 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:49.144712 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 29 01:04:50.144588 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:50.144544 2558 patch_prober.go:28] interesting pod/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 29 01:04:50.144986 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:50.144613 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 29 01:04:51.144404 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:51.144373 2558 patch_prober.go:28] interesting pod/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 29 01:04:51.144573 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:51.144427 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 29 01:04:52.144335 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:52.144276 2558 patch_prober.go:28] interesting pod/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 29 01:04:52.144743 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:52.144365 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 29 01:04:53.144040 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:53.144007 2558 patch_prober.go:28] interesting pod/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 29 01:04:53.144213 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:53.144065 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 29 01:04:54.148334 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:54.148287 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:54.148791 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:54.148644 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:04:54.149368 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:04:54.149351 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:05:04.922767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:04.922726 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62"] Jul 29 01:05:04.923254 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:04.922990 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerName="istio-proxy" containerID="cri-o://64809ce223785a4c999acd96579949406c38a16000992160e80ca40af69d1e75" gracePeriod=30 Jul 29 01:05:05.664530 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.664499 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m"] Jul 29 01:05:05.664883 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.664871 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="487904a4-c22c-4544-9506-f4582b02b144" containerName="istio-proxy" Jul 29 01:05:05.664929 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.664885 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="487904a4-c22c-4544-9506-f4582b02b144" containerName="istio-proxy" Jul 29 01:05:05.664929 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.664895 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="487904a4-c22c-4544-9506-f4582b02b144" containerName="istio-proxy" Jul 29 01:05:05.664929 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.664900 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="487904a4-c22c-4544-9506-f4582b02b144" containerName="istio-proxy" Jul 29 01:05:05.665019 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.664958 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="487904a4-c22c-4544-9506-f4582b02b144" containerName="istio-proxy" Jul 29 01:05:05.665019 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.664966 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="487904a4-c22c-4544-9506-f4582b02b144" containerName="istio-proxy" Jul 29 01:05:05.668140 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.668118 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.669913 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.669890 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c63818d3-openshift-default-dockercfg-jf4tx\"" Jul 29 01:05:05.670099 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.670073 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-podinfo\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.670166 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.670140 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-envoy\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.670218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.670169 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg5s5\" (UniqueName: \"kubernetes.io/projected/9582865b-3fca-4f3e-a155-8dff3dc6dde4-kube-api-access-rg5s5\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.670278 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.670240 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istiod-ca-cert\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.670278 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.670270 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-token\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.670413 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.670303 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-workload-certs\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.670413 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.670356 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-data\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.670413 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.670392 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-workload-socket\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.670554 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.670416 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-credential-socket\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.678634 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.678609 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m"] Jul 29 01:05:05.771559 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.771526 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istiod-ca-cert\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.771716 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.771569 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-token\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.771801 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.771743 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-workload-certs\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.771856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.771798 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-data\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.771856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.771843 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-workload-socket\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.771960 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.771869 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-credential-socket\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.771960 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.771902 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-podinfo\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.772070 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.771988 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-envoy\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.772070 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.772020 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg5s5\" (UniqueName: \"kubernetes.io/projected/9582865b-3fca-4f3e-a155-8dff3dc6dde4-kube-api-access-rg5s5\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.772233 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.772209 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-workload-certs\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.772435 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.772337 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-data\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.772435 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.772422 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istiod-ca-cert\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.772580 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.772487 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-credential-socket\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.772686 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.772654 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-workload-socket\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.775402 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.775375 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-podinfo\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.775628 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.775610 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-envoy\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.778950 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.778882 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-token\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.779481 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.779447 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg5s5\" (UniqueName: \"kubernetes.io/projected/9582865b-3fca-4f3e-a155-8dff3dc6dde4-kube-api-access-rg5s5\") pod \"e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:05.980871 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:05.980792 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:06.118987 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:06.118915 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m"] Jul 29 01:05:06.121729 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:05:06.121700 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9582865b_3fca_4f3e_a155_8dff3dc6dde4.slice/crio-819088fa5a2ba166b1727e13f6f422e51bc4e9f6c0b515ab90156baa4c999ef4 WatchSource:0}: Error finding container 819088fa5a2ba166b1727e13f6f422e51bc4e9f6c0b515ab90156baa4c999ef4: Status 404 returned error can't find the container with id 819088fa5a2ba166b1727e13f6f422e51bc4e9f6c0b515ab90156baa4c999ef4 Jul 29 01:05:06.123906 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:06.123873 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:05:06.124007 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:06.123940 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:05:06.124007 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:06.123969 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:05:06.975043 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:06.975006 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" event={"ID":"9582865b-3fca-4f3e-a155-8dff3dc6dde4","Type":"ContainerStarted","Data":"bc51d570c19afccd1672dba9cf36fd1110adf63c928db524cf4d4942f3712e2e"} Jul 29 01:05:06.975220 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:06.975070 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" event={"ID":"9582865b-3fca-4f3e-a155-8dff3dc6dde4","Type":"ContainerStarted","Data":"819088fa5a2ba166b1727e13f6f422e51bc4e9f6c0b515ab90156baa4c999ef4"} Jul 29 01:05:06.981879 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:06.981854 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:06.994472 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:06.994430 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" podStartSLOduration=1.994416759 podStartE2EDuration="1.994416759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:05:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:05:06.992397229 +0000 UTC m=+1758.651623112" watchObservedRunningTime="2026-07-29 01:05:06.994416759 +0000 UTC m=+1758.653642656" Jul 29 01:05:07.986920 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:07.986891 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:08.985972 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:08.985935 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:08.987002 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:08.986978 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:09.150927 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:09.150894 2558 patch_prober.go:28] interesting pod/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 01:05:09.151097 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:09.150962 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 01:05:10.169612 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.169586 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:05:10.210428 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.210347 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istiod-ca-cert\") pod \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " Jul 29 01:05:10.210428 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.210395 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnsv2\" (UniqueName: \"kubernetes.io/projected/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-kube-api-access-pnsv2\") pod \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " Jul 29 01:05:10.210428 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.210423 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-workload-certs\") pod \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " Jul 29 01:05:10.210688 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.210446 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-envoy\") pod \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " Jul 29 01:05:10.210688 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.210496 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-credential-socket\") pod \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " Jul 29 01:05:10.210688 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.210522 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-token\") pod \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " Jul 29 01:05:10.210688 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.210547 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-data\") pod \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " Jul 29 01:05:10.210688 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.210595 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-workload-socket\") pod \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " Jul 29 01:05:10.210688 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.210619 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-podinfo\") pod \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\" (UID: \"6c5762a7-aaeb-48b3-9028-2f24526cf2e8\") " Jul 29 01:05:10.211047 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.210759 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6c5762a7-aaeb-48b3-9028-2f24526cf2e8" (UID: "6c5762a7-aaeb-48b3-9028-2f24526cf2e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:05:10.211047 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.210815 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6c5762a7-aaeb-48b3-9028-2f24526cf2e8" (UID: "6c5762a7-aaeb-48b3-9028-2f24526cf2e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:05:10.211047 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.210972 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6c5762a7-aaeb-48b3-9028-2f24526cf2e8" (UID: "6c5762a7-aaeb-48b3-9028-2f24526cf2e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:05:10.211224 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.211061 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istiod-ca-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:10.211224 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.211080 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-workload-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:10.211224 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.211093 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-credential-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:10.211224 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.211171 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6c5762a7-aaeb-48b3-9028-2f24526cf2e8" (UID: "6c5762a7-aaeb-48b3-9028-2f24526cf2e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:05:10.211744 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.211706 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "6c5762a7-aaeb-48b3-9028-2f24526cf2e8" (UID: "6c5762a7-aaeb-48b3-9028-2f24526cf2e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:05:10.212866 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.212846 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-kube-api-access-pnsv2" (OuterVolumeSpecName: "kube-api-access-pnsv2") pod "6c5762a7-aaeb-48b3-9028-2f24526cf2e8" (UID: "6c5762a7-aaeb-48b3-9028-2f24526cf2e8"). InnerVolumeSpecName "kube-api-access-pnsv2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:05:10.213046 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.213023 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6c5762a7-aaeb-48b3-9028-2f24526cf2e8" (UID: "6c5762a7-aaeb-48b3-9028-2f24526cf2e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 01:05:10.213227 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.213204 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6c5762a7-aaeb-48b3-9028-2f24526cf2e8" (UID: "6c5762a7-aaeb-48b3-9028-2f24526cf2e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:05:10.213349 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.213230 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "6c5762a7-aaeb-48b3-9028-2f24526cf2e8" (UID: "6c5762a7-aaeb-48b3-9028-2f24526cf2e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:05:10.312264 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.312230 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnsv2\" (UniqueName: \"kubernetes.io/projected/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-kube-api-access-pnsv2\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:10.312264 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.312258 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-envoy\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:10.312264 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.312268 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:10.312512 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.312276 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-data\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:10.312512 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.312284 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-workload-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:10.312512 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.312292 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c5762a7-aaeb-48b3-9028-2f24526cf2e8-istio-podinfo\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:10.999285 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.999248 2558 generic.go:356] "Generic (PLEG): container finished" podID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerID="64809ce223785a4c999acd96579949406c38a16000992160e80ca40af69d1e75" exitCode=0 Jul 29 01:05:10.999464 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.999293 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" Jul 29 01:05:10.999464 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.999348 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" event={"ID":"6c5762a7-aaeb-48b3-9028-2f24526cf2e8","Type":"ContainerDied","Data":"64809ce223785a4c999acd96579949406c38a16000992160e80ca40af69d1e75"} Jul 29 01:05:10.999464 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.999402 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62" event={"ID":"6c5762a7-aaeb-48b3-9028-2f24526cf2e8","Type":"ContainerDied","Data":"b6f1ab65559a0b9f6c3b458e768402b30b994c98a786dd68816e60225deaeee6"} Jul 29 01:05:10.999464 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:10.999425 2558 scope.go:117] "RemoveContainer" containerID="64809ce223785a4c999acd96579949406c38a16000992160e80ca40af69d1e75" Jul 29 01:05:11.008397 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:11.008380 2558 scope.go:117] "RemoveContainer" containerID="a0f28135e460a0b5eef642123e3ca73c6b2fb4bcdf900b9056cf86ee32a19dce" Jul 29 01:05:11.014282 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:11.014262 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62"] Jul 29 01:05:11.016674 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:11.016657 2558 scope.go:117] "RemoveContainer" containerID="64809ce223785a4c999acd96579949406c38a16000992160e80ca40af69d1e75" Jul 29 01:05:11.016923 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:05:11.016903 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64809ce223785a4c999acd96579949406c38a16000992160e80ca40af69d1e75\": container with ID starting with 64809ce223785a4c999acd96579949406c38a16000992160e80ca40af69d1e75 not found: ID does not exist" containerID="64809ce223785a4c999acd96579949406c38a16000992160e80ca40af69d1e75" Jul 29 01:05:11.016992 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:11.016936 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64809ce223785a4c999acd96579949406c38a16000992160e80ca40af69d1e75"} err="failed to get container status \"64809ce223785a4c999acd96579949406c38a16000992160e80ca40af69d1e75\": rpc error: code = NotFound desc = could not find container \"64809ce223785a4c999acd96579949406c38a16000992160e80ca40af69d1e75\": container with ID starting with 64809ce223785a4c999acd96579949406c38a16000992160e80ca40af69d1e75 not found: ID does not exist" Jul 29 01:05:11.016992 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:11.016961 2558 scope.go:117] "RemoveContainer" containerID="a0f28135e460a0b5eef642123e3ca73c6b2fb4bcdf900b9056cf86ee32a19dce" Jul 29 01:05:11.017197 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:05:11.017180 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0f28135e460a0b5eef642123e3ca73c6b2fb4bcdf900b9056cf86ee32a19dce\": container with ID starting with a0f28135e460a0b5eef642123e3ca73c6b2fb4bcdf900b9056cf86ee32a19dce not found: ID does not exist" containerID="a0f28135e460a0b5eef642123e3ca73c6b2fb4bcdf900b9056cf86ee32a19dce" Jul 29 01:05:11.017240 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:11.017203 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0f28135e460a0b5eef642123e3ca73c6b2fb4bcdf900b9056cf86ee32a19dce"} err="failed to get container status \"a0f28135e460a0b5eef642123e3ca73c6b2fb4bcdf900b9056cf86ee32a19dce\": rpc error: code = NotFound desc = could not find container \"a0f28135e460a0b5eef642123e3ca73c6b2fb4bcdf900b9056cf86ee32a19dce\": container with ID starting with a0f28135e460a0b5eef642123e3ca73c6b2fb4bcdf900b9056cf86ee32a19dce not found: ID does not exist" Jul 29 01:05:11.020511 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:11.020492 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-956869dd-gw-openshift-default-547b9544d-4js62"] Jul 29 01:05:12.789711 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:12.789670 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" path="/var/lib/kubelet/pods/6c5762a7-aaeb-48b3-9028-2f24526cf2e8/volumes" Jul 29 01:05:23.987734 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:23.987701 2558 patch_prober.go:28] interesting pod/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 29 01:05:23.988133 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:23.987763 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" podUID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 29 01:05:24.068446 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:24.068408 2558 generic.go:356] "Generic (PLEG): container finished" podID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" containerID="bc51d570c19afccd1672dba9cf36fd1110adf63c928db524cf4d4942f3712e2e" exitCode=0 Jul 29 01:05:24.068603 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:24.068480 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" event={"ID":"9582865b-3fca-4f3e-a155-8dff3dc6dde4","Type":"ContainerDied","Data":"bc51d570c19afccd1672dba9cf36fd1110adf63c928db524cf4d4942f3712e2e"} Jul 29 01:05:24.068963 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:24.068944 2558 scope.go:117] "RemoveContainer" containerID="bc51d570c19afccd1672dba9cf36fd1110adf63c928db524cf4d4942f3712e2e" Jul 29 01:05:24.070113 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:24.070080 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:05:24.070218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:24.070150 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:05:24.070218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:24.070182 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:05:25.084703 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:25.084667 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" event={"ID":"9582865b-3fca-4f3e-a155-8dff3dc6dde4","Type":"ContainerStarted","Data":"1e679c0184f23d41bcfa7531638e3e8ea8ef4948797640c3b4c5ebdd0b7efd1e"} Jul 29 01:05:25.981692 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:25.981659 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:25.981866 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:25.981836 2558 patch_prober.go:28] interesting pod/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 29 01:05:25.981910 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:25.981897 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" podUID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 29 01:05:26.981629 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:26.981584 2558 patch_prober.go:28] interesting pod/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 29 01:05:26.982401 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:26.981640 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" podUID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 29 01:05:27.981540 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:27.981503 2558 patch_prober.go:28] interesting pod/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 29 01:05:27.981713 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:27.981567 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" podUID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 29 01:05:28.985044 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:28.985012 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:28.985492 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:28.985345 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:28.986040 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:28.986020 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:40.194302 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:40.194271 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m"] Jul 29 01:05:40.194787 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:40.194546 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" podUID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" containerName="istio-proxy" containerID="cri-o://1e679c0184f23d41bcfa7531638e3e8ea8ef4948797640c3b4c5ebdd0b7efd1e" gracePeriod=30 Jul 29 01:05:40.917665 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:40.917625 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4"] Jul 29 01:05:40.918049 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:40.918034 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerName="istio-proxy" Jul 29 01:05:40.918116 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:40.918050 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerName="istio-proxy" Jul 29 01:05:40.918116 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:40.918064 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerName="istio-proxy" Jul 29 01:05:40.918116 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:40.918070 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerName="istio-proxy" Jul 29 01:05:40.918213 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:40.918136 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerName="istio-proxy" Jul 29 01:05:40.918213 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:40.918146 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c5762a7-aaeb-48b3-9028-2f24526cf2e8" containerName="istio-proxy" Jul 29 01:05:40.920352 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:40.920331 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:40.922364 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:40.922344 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-030a9f65-openshift-default-dockercfg-hrr74\"" Jul 29 01:05:40.932792 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:40.932766 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4"] Jul 29 01:05:41.087707 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.087669 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istiod-ca-cert\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.087874 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.087752 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-workload-certs\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.087874 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.087779 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-token\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.087874 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.087807 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-workload-socket\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.087998 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.087888 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-podinfo\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.087998 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.087976 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-envoy\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.088074 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.087996 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-data\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.088074 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.088014 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-credential-socket\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.088074 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.088043 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwjlv\" (UniqueName: \"kubernetes.io/projected/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-kube-api-access-nwjlv\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.188790 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.188699 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-envoy\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.188790 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.188739 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-data\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.188790 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.188763 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-credential-socket\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.188790 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.188787 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwjlv\" (UniqueName: \"kubernetes.io/projected/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-kube-api-access-nwjlv\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.189125 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.189099 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istiod-ca-cert\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.189194 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.189125 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-data\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.189253 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.189200 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-credential-socket\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.189388 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.189364 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-workload-certs\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.189445 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.189421 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-token\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.189481 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.189452 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-workload-socket\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.189481 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.189476 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-podinfo\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.189698 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.189666 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-workload-certs\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.189781 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.189762 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-workload-socket\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.189866 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.189847 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istiod-ca-cert\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.191294 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.191270 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-envoy\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.191973 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.191945 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-podinfo\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.201552 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.201531 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-token\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.201826 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.201746 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwjlv\" (UniqueName: \"kubernetes.io/projected/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-kube-api-access-nwjlv\") pod \"e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.231286 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.231264 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:41.367647 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.367619 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4"] Jul 29 01:05:41.369284 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:05:41.369257 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f0d64c8_751d_4e55_bdf2_91b47c885ef9.slice/crio-bc8ae4a999b1c9ebebcfade13a5b1d5d59991ab91918f6bacd5489d7c648665e WatchSource:0}: Error finding container bc8ae4a999b1c9ebebcfade13a5b1d5d59991ab91918f6bacd5489d7c648665e: Status 404 returned error can't find the container with id bc8ae4a999b1c9ebebcfade13a5b1d5d59991ab91918f6bacd5489d7c648665e Jul 29 01:05:41.371416 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.371383 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:05:41.371495 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.371448 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:05:41.371495 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:41.371477 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:05:42.166746 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:42.166704 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" event={"ID":"9f0d64c8-751d-4e55-bdf2-91b47c885ef9","Type":"ContainerStarted","Data":"9ca193ccdb67a15ade8e87ca78e4f6787a53c46ea39ccb283851c96d4b43224a"} Jul 29 01:05:42.166911 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:42.166767 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" event={"ID":"9f0d64c8-751d-4e55-bdf2-91b47c885ef9","Type":"ContainerStarted","Data":"bc8ae4a999b1c9ebebcfade13a5b1d5d59991ab91918f6bacd5489d7c648665e"} Jul 29 01:05:42.185688 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:42.185636 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" podStartSLOduration=2.185620234 podStartE2EDuration="2.185620234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:05:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:05:42.185196771 +0000 UTC m=+1793.844422678" watchObservedRunningTime="2026-07-29 01:05:42.185620234 +0000 UTC m=+1793.844846127" Jul 29 01:05:42.231787 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:42.231756 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:42.236743 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:42.236720 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:43.171570 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:43.171536 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:43.172537 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:43.172516 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:05:43.987901 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:43.987860 2558 patch_prober.go:28] interesting pod/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 01:05:43.988260 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:43.987923 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" podUID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 01:05:45.360950 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.360927 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:45.529124 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529037 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-podinfo\") pod \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " Jul 29 01:05:45.529124 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529079 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-data\") pod \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " Jul 29 01:05:45.529124 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529123 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istiod-ca-cert\") pod \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " Jul 29 01:05:45.529406 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529142 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg5s5\" (UniqueName: \"kubernetes.io/projected/9582865b-3fca-4f3e-a155-8dff3dc6dde4-kube-api-access-rg5s5\") pod \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " Jul 29 01:05:45.529406 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529157 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-workload-socket\") pod \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " Jul 29 01:05:45.529406 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529198 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-workload-certs\") pod \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " Jul 29 01:05:45.529406 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529230 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-envoy\") pod \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " Jul 29 01:05:45.529406 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529258 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-credential-socket\") pod \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " Jul 29 01:05:45.529406 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529358 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-token\") pod \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\" (UID: \"9582865b-3fca-4f3e-a155-8dff3dc6dde4\") " Jul 29 01:05:45.529715 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529515 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9582865b-3fca-4f3e-a155-8dff3dc6dde4" (UID: "9582865b-3fca-4f3e-a155-8dff3dc6dde4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:05:45.529715 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529529 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9582865b-3fca-4f3e-a155-8dff3dc6dde4" (UID: "9582865b-3fca-4f3e-a155-8dff3dc6dde4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:05:45.529715 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529592 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9582865b-3fca-4f3e-a155-8dff3dc6dde4" (UID: "9582865b-3fca-4f3e-a155-8dff3dc6dde4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:05:45.529715 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529698 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9582865b-3fca-4f3e-a155-8dff3dc6dde4" (UID: "9582865b-3fca-4f3e-a155-8dff3dc6dde4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:05:45.529928 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529760 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istiod-ca-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:45.529928 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529782 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-workload-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:45.529928 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.529795 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-workload-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:45.530118 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.530091 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-data" (OuterVolumeSpecName: "istio-data") pod "9582865b-3fca-4f3e-a155-8dff3dc6dde4" (UID: "9582865b-3fca-4f3e-a155-8dff3dc6dde4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:05:45.531520 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.531500 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-token" (OuterVolumeSpecName: "istio-token") pod "9582865b-3fca-4f3e-a155-8dff3dc6dde4" (UID: "9582865b-3fca-4f3e-a155-8dff3dc6dde4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:05:45.531598 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.531565 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9582865b-3fca-4f3e-a155-8dff3dc6dde4-kube-api-access-rg5s5" (OuterVolumeSpecName: "kube-api-access-rg5s5") pod "9582865b-3fca-4f3e-a155-8dff3dc6dde4" (UID: "9582865b-3fca-4f3e-a155-8dff3dc6dde4"). InnerVolumeSpecName "kube-api-access-rg5s5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:05:45.531711 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.531689 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9582865b-3fca-4f3e-a155-8dff3dc6dde4" (UID: "9582865b-3fca-4f3e-a155-8dff3dc6dde4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:05:45.531711 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.531700 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9582865b-3fca-4f3e-a155-8dff3dc6dde4" (UID: "9582865b-3fca-4f3e-a155-8dff3dc6dde4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 01:05:45.630785 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.630748 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:45.630785 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.630783 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-podinfo\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:45.630785 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.630792 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-data\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:45.630992 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.630800 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg5s5\" (UniqueName: \"kubernetes.io/projected/9582865b-3fca-4f3e-a155-8dff3dc6dde4-kube-api-access-rg5s5\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:45.630992 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.630810 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-istio-envoy\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:45.630992 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:45.630819 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9582865b-3fca-4f3e-a155-8dff3dc6dde4-credential-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:05:46.188461 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:46.188430 2558 generic.go:356] "Generic (PLEG): container finished" podID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" containerID="1e679c0184f23d41bcfa7531638e3e8ea8ef4948797640c3b4c5ebdd0b7efd1e" exitCode=0 Jul 29 01:05:46.188624 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:46.188476 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" Jul 29 01:05:46.188624 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:46.188509 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" event={"ID":"9582865b-3fca-4f3e-a155-8dff3dc6dde4","Type":"ContainerDied","Data":"1e679c0184f23d41bcfa7531638e3e8ea8ef4948797640c3b4c5ebdd0b7efd1e"} Jul 29 01:05:46.188624 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:46.188563 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m" event={"ID":"9582865b-3fca-4f3e-a155-8dff3dc6dde4","Type":"ContainerDied","Data":"819088fa5a2ba166b1727e13f6f422e51bc4e9f6c0b515ab90156baa4c999ef4"} Jul 29 01:05:46.188624 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:46.188587 2558 scope.go:117] "RemoveContainer" containerID="1e679c0184f23d41bcfa7531638e3e8ea8ef4948797640c3b4c5ebdd0b7efd1e" Jul 29 01:05:46.198083 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:46.198062 2558 scope.go:117] "RemoveContainer" containerID="bc51d570c19afccd1672dba9cf36fd1110adf63c928db524cf4d4942f3712e2e" Jul 29 01:05:46.208821 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:46.207200 2558 scope.go:117] "RemoveContainer" containerID="1e679c0184f23d41bcfa7531638e3e8ea8ef4948797640c3b4c5ebdd0b7efd1e" Jul 29 01:05:46.209354 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:05:46.209331 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e679c0184f23d41bcfa7531638e3e8ea8ef4948797640c3b4c5ebdd0b7efd1e\": container with ID starting with 1e679c0184f23d41bcfa7531638e3e8ea8ef4948797640c3b4c5ebdd0b7efd1e not found: ID does not exist" containerID="1e679c0184f23d41bcfa7531638e3e8ea8ef4948797640c3b4c5ebdd0b7efd1e" Jul 29 01:05:46.209453 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:46.209364 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e679c0184f23d41bcfa7531638e3e8ea8ef4948797640c3b4c5ebdd0b7efd1e"} err="failed to get container status \"1e679c0184f23d41bcfa7531638e3e8ea8ef4948797640c3b4c5ebdd0b7efd1e\": rpc error: code = NotFound desc = could not find container \"1e679c0184f23d41bcfa7531638e3e8ea8ef4948797640c3b4c5ebdd0b7efd1e\": container with ID starting with 1e679c0184f23d41bcfa7531638e3e8ea8ef4948797640c3b4c5ebdd0b7efd1e not found: ID does not exist" Jul 29 01:05:46.209453 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:46.209387 2558 scope.go:117] "RemoveContainer" containerID="bc51d570c19afccd1672dba9cf36fd1110adf63c928db524cf4d4942f3712e2e" Jul 29 01:05:46.209846 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:05:46.209822 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc51d570c19afccd1672dba9cf36fd1110adf63c928db524cf4d4942f3712e2e\": container with ID starting with bc51d570c19afccd1672dba9cf36fd1110adf63c928db524cf4d4942f3712e2e not found: ID does not exist" containerID="bc51d570c19afccd1672dba9cf36fd1110adf63c928db524cf4d4942f3712e2e" Jul 29 01:05:46.209942 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:46.209855 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc51d570c19afccd1672dba9cf36fd1110adf63c928db524cf4d4942f3712e2e"} err="failed to get container status \"bc51d570c19afccd1672dba9cf36fd1110adf63c928db524cf4d4942f3712e2e\": rpc error: code = NotFound desc = could not find container \"bc51d570c19afccd1672dba9cf36fd1110adf63c928db524cf4d4942f3712e2e\": container with ID starting with bc51d570c19afccd1672dba9cf36fd1110adf63c928db524cf4d4942f3712e2e not found: ID does not exist" Jul 29 01:05:46.211060 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:46.211039 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m"] Jul 29 01:05:46.212776 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:46.212757 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c63818d3-openshift-default-5dc47cfb84-7mh6m"] Jul 29 01:05:46.788976 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:46.788944 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" path="/var/lib/kubelet/pods/9582865b-3fca-4f3e-a155-8dff3dc6dde4/volumes" Jul 29 01:05:48.860136 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:48.860106 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:05:48.864218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:48.864192 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:05:48.864858 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:48.864838 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 01:05:48.868895 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:48.868875 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 01:05:58.172791 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:58.172757 2558 patch_prober.go:28] interesting pod/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 29 01:05:58.173186 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:58.172822 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" podUID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 29 01:05:58.252281 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:58.252246 2558 generic.go:356] "Generic (PLEG): container finished" podID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" containerID="9ca193ccdb67a15ade8e87ca78e4f6787a53c46ea39ccb283851c96d4b43224a" exitCode=0 Jul 29 01:05:58.252448 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:58.252335 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" event={"ID":"9f0d64c8-751d-4e55-bdf2-91b47c885ef9","Type":"ContainerDied","Data":"9ca193ccdb67a15ade8e87ca78e4f6787a53c46ea39ccb283851c96d4b43224a"} Jul 29 01:05:58.252723 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:58.252708 2558 scope.go:117] "RemoveContainer" containerID="9ca193ccdb67a15ade8e87ca78e4f6787a53c46ea39ccb283851c96d4b43224a" Jul 29 01:05:58.253740 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:58.253708 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:05:58.253815 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:58.253799 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:05:58.253853 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:58.253839 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:05:59.260863 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:05:59.260812 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" event={"ID":"9f0d64c8-751d-4e55-bdf2-91b47c885ef9","Type":"ContainerStarted","Data":"6a0e5295a0e36fb6518056deb518ddf51551cc3d18d70fe6108d18f86ce29720"} Jul 29 01:06:00.231922 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:00.231889 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:06:00.232156 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:00.232137 2558 patch_prober.go:28] interesting pod/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 29 01:06:00.232215 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:00.232180 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" podUID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 29 01:06:01.231977 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:01.231942 2558 patch_prober.go:28] interesting pod/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 29 01:06:01.232384 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:01.232008 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" podUID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 29 01:06:02.232251 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:02.232212 2558 patch_prober.go:28] interesting pod/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 29 01:06:02.232663 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:02.232283 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" podUID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 29 01:06:03.236579 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:03.236545 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:06:03.236978 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:03.236954 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:06:03.237613 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:03.237593 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:06:14.425333 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:14.411975 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4"] Jul 29 01:06:14.426178 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:14.426123 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" podUID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" containerName="istio-proxy" containerID="cri-o://6a0e5295a0e36fb6518056deb518ddf51551cc3d18d70fe6108d18f86ce29720" gracePeriod=30 Jul 29 01:06:18.239554 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:18.239516 2558 patch_prober.go:28] interesting pod/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 01:06:18.239989 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:18.239597 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" podUID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 01:06:19.675598 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.675572 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:06:19.839803 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.839776 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-workload-socket\") pod \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " Jul 29 01:06:19.839984 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.839814 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-podinfo\") pod \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " Jul 29 01:06:19.839984 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.839854 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-workload-certs\") pod \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " Jul 29 01:06:19.839984 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.839881 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-credential-socket\") pod \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " Jul 29 01:06:19.839984 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.839900 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-token\") pod \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " Jul 29 01:06:19.839984 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.839914 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-envoy\") pod \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " Jul 29 01:06:19.839984 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.839952 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-data\") pod \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " Jul 29 01:06:19.840385 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.840012 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwjlv\" (UniqueName: \"kubernetes.io/projected/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-kube-api-access-nwjlv\") pod \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " Jul 29 01:06:19.840385 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.840062 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istiod-ca-cert\") pod \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\" (UID: \"9f0d64c8-751d-4e55-bdf2-91b47c885ef9\") " Jul 29 01:06:19.840385 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.840096 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9f0d64c8-751d-4e55-bdf2-91b47c885ef9" (UID: "9f0d64c8-751d-4e55-bdf2-91b47c885ef9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:06:19.840385 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.840236 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9f0d64c8-751d-4e55-bdf2-91b47c885ef9" (UID: "9f0d64c8-751d-4e55-bdf2-91b47c885ef9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:06:19.840385 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.840271 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9f0d64c8-751d-4e55-bdf2-91b47c885ef9" (UID: "9f0d64c8-751d-4e55-bdf2-91b47c885ef9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:06:19.840647 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.840397 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-workload-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:06:19.840647 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.840418 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-workload-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:06:19.840647 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.840433 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-credential-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:06:19.840647 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.840548 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9f0d64c8-751d-4e55-bdf2-91b47c885ef9" (UID: "9f0d64c8-751d-4e55-bdf2-91b47c885ef9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:06:19.840885 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.840865 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-data" (OuterVolumeSpecName: "istio-data") pod "9f0d64c8-751d-4e55-bdf2-91b47c885ef9" (UID: "9f0d64c8-751d-4e55-bdf2-91b47c885ef9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:06:19.842214 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.842189 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-token" (OuterVolumeSpecName: "istio-token") pod "9f0d64c8-751d-4e55-bdf2-91b47c885ef9" (UID: "9f0d64c8-751d-4e55-bdf2-91b47c885ef9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:06:19.842214 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.842197 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-kube-api-access-nwjlv" (OuterVolumeSpecName: "kube-api-access-nwjlv") pod "9f0d64c8-751d-4e55-bdf2-91b47c885ef9" (UID: "9f0d64c8-751d-4e55-bdf2-91b47c885ef9"). InnerVolumeSpecName "kube-api-access-nwjlv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:06:19.842429 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.842412 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9f0d64c8-751d-4e55-bdf2-91b47c885ef9" (UID: "9f0d64c8-751d-4e55-bdf2-91b47c885ef9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:06:19.842470 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.842442 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9f0d64c8-751d-4e55-bdf2-91b47c885ef9" (UID: "9f0d64c8-751d-4e55-bdf2-91b47c885ef9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 01:06:19.941210 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.941172 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwjlv\" (UniqueName: \"kubernetes.io/projected/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-kube-api-access-nwjlv\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:06:19.941210 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.941210 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istiod-ca-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:06:19.941425 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.941224 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-podinfo\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:06:19.941425 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.941237 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:06:19.941425 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.941262 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-envoy\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:06:19.941425 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:19.941274 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f0d64c8-751d-4e55-bdf2-91b47c885ef9-istio-data\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:06:20.372888 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:20.372848 2558 generic.go:356] "Generic (PLEG): container finished" podID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" containerID="6a0e5295a0e36fb6518056deb518ddf51551cc3d18d70fe6108d18f86ce29720" exitCode=0 Jul 29 01:06:20.373065 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:20.372897 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" Jul 29 01:06:20.373065 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:20.372892 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" event={"ID":"9f0d64c8-751d-4e55-bdf2-91b47c885ef9","Type":"ContainerDied","Data":"6a0e5295a0e36fb6518056deb518ddf51551cc3d18d70fe6108d18f86ce29720"} Jul 29 01:06:20.373065 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:20.373021 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4" event={"ID":"9f0d64c8-751d-4e55-bdf2-91b47c885ef9","Type":"ContainerDied","Data":"bc8ae4a999b1c9ebebcfade13a5b1d5d59991ab91918f6bacd5489d7c648665e"} Jul 29 01:06:20.373065 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:20.373054 2558 scope.go:117] "RemoveContainer" containerID="6a0e5295a0e36fb6518056deb518ddf51551cc3d18d70fe6108d18f86ce29720" Jul 29 01:06:20.382489 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:20.382468 2558 scope.go:117] "RemoveContainer" containerID="9ca193ccdb67a15ade8e87ca78e4f6787a53c46ea39ccb283851c96d4b43224a" Jul 29 01:06:20.390807 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:20.390791 2558 scope.go:117] "RemoveContainer" containerID="6a0e5295a0e36fb6518056deb518ddf51551cc3d18d70fe6108d18f86ce29720" Jul 29 01:06:20.391039 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:06:20.391021 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a0e5295a0e36fb6518056deb518ddf51551cc3d18d70fe6108d18f86ce29720\": container with ID starting with 6a0e5295a0e36fb6518056deb518ddf51551cc3d18d70fe6108d18f86ce29720 not found: ID does not exist" containerID="6a0e5295a0e36fb6518056deb518ddf51551cc3d18d70fe6108d18f86ce29720" Jul 29 01:06:20.391084 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:20.391047 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a0e5295a0e36fb6518056deb518ddf51551cc3d18d70fe6108d18f86ce29720"} err="failed to get container status \"6a0e5295a0e36fb6518056deb518ddf51551cc3d18d70fe6108d18f86ce29720\": rpc error: code = NotFound desc = could not find container \"6a0e5295a0e36fb6518056deb518ddf51551cc3d18d70fe6108d18f86ce29720\": container with ID starting with 6a0e5295a0e36fb6518056deb518ddf51551cc3d18d70fe6108d18f86ce29720 not found: ID does not exist" Jul 29 01:06:20.391084 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:20.391063 2558 scope.go:117] "RemoveContainer" containerID="9ca193ccdb67a15ade8e87ca78e4f6787a53c46ea39ccb283851c96d4b43224a" Jul 29 01:06:20.391276 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:06:20.391258 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ca193ccdb67a15ade8e87ca78e4f6787a53c46ea39ccb283851c96d4b43224a\": container with ID starting with 9ca193ccdb67a15ade8e87ca78e4f6787a53c46ea39ccb283851c96d4b43224a not found: ID does not exist" containerID="9ca193ccdb67a15ade8e87ca78e4f6787a53c46ea39ccb283851c96d4b43224a" Jul 29 01:06:20.391360 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:20.391279 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ca193ccdb67a15ade8e87ca78e4f6787a53c46ea39ccb283851c96d4b43224a"} err="failed to get container status \"9ca193ccdb67a15ade8e87ca78e4f6787a53c46ea39ccb283851c96d4b43224a\": rpc error: code = NotFound desc = could not find container \"9ca193ccdb67a15ade8e87ca78e4f6787a53c46ea39ccb283851c96d4b43224a\": container with ID starting with 9ca193ccdb67a15ade8e87ca78e4f6787a53c46ea39ccb283851c96d4b43224a not found: ID does not exist" Jul 29 01:06:20.394370 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:20.394350 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4"] Jul 29 01:06:20.398523 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:20.398504 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-030a9f65-openshift-default-66bf9d6d5b-svqz4"] Jul 29 01:06:20.790769 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:06:20.790694 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" path="/var/lib/kubelet/pods/9f0d64c8-751d-4e55-bdf2-91b47c885ef9/volumes" Jul 29 01:07:56.307633 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.307597 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt"] Jul 29 01:07:56.308218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.307978 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" containerName="istio-proxy" Jul 29 01:07:56.308218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.307989 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" containerName="istio-proxy" Jul 29 01:07:56.308218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.308001 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" containerName="istio-proxy" Jul 29 01:07:56.308218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.308007 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" containerName="istio-proxy" Jul 29 01:07:56.308218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.308020 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" containerName="istio-proxy" Jul 29 01:07:56.308218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.308025 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" containerName="istio-proxy" Jul 29 01:07:56.308218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.308034 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" containerName="istio-proxy" Jul 29 01:07:56.308218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.308039 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" containerName="istio-proxy" Jul 29 01:07:56.308218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.308099 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" containerName="istio-proxy" Jul 29 01:07:56.308218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.308107 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" containerName="istio-proxy" Jul 29 01:07:56.308218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.308113 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9582865b-3fca-4f3e-a155-8dff3dc6dde4" containerName="istio-proxy" Jul 29 01:07:56.311039 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.311020 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.313027 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.313009 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b8aa042b-openshift-default-dockercfg-dt2wf\"" Jul 29 01:07:56.330222 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.330200 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt"] Jul 29 01:07:56.354322 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.354282 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-podinfo\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.354420 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.354340 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istiod-ca-cert\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.354465 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.354421 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-data\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.354513 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.354470 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-envoy\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.354555 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.354534 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6hdv\" (UniqueName: \"kubernetes.io/projected/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-kube-api-access-c6hdv\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.354596 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.354570 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-workload-socket\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.354641 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.354599 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-credential-socket\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.354641 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.354620 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-workload-certs\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.354719 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.354706 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-token\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.455667 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.455629 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-workload-socket\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.455831 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.455678 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-credential-socket\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.455831 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.455707 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-workload-certs\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.455831 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.455748 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-token\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.455831 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.455797 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-podinfo\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.455831 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.455820 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istiod-ca-cert\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.456092 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.455852 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-data\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.456092 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.455887 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-envoy\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.456092 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.455933 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6hdv\" (UniqueName: \"kubernetes.io/projected/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-kube-api-access-c6hdv\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.456092 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.456072 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-workload-socket\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.456276 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.456123 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-credential-socket\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.456276 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.456148 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-workload-certs\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.456429 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.456407 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-data\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.456750 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.456718 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istiod-ca-cert\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.458364 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.458344 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-envoy\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.458686 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.458665 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-podinfo\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.469953 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.469931 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-token\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.470045 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.470036 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6hdv\" (UniqueName: \"kubernetes.io/projected/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-kube-api-access-c6hdv\") pod \"e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.621479 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.621446 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:56.767488 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.767461 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt"] Jul 29 01:07:56.769283 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:07:56.769257 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaaec969_fdff_4ff3_ac0f_a68c83f8a233.slice/crio-b5074defee7c1eb166e0eda98f9f3566c850e25bb63c59ce82cc194678d371b2 WatchSource:0}: Error finding container b5074defee7c1eb166e0eda98f9f3566c850e25bb63c59ce82cc194678d371b2: Status 404 returned error can't find the container with id b5074defee7c1eb166e0eda98f9f3566c850e25bb63c59ce82cc194678d371b2 Jul 29 01:07:56.771278 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.771243 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:07:56.771414 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.771341 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:07:56.771414 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.771392 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:07:56.818360 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:56.818328 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" event={"ID":"eaaec969-fdff-4ff3-ac0f-a68c83f8a233","Type":"ContainerStarted","Data":"b5074defee7c1eb166e0eda98f9f3566c850e25bb63c59ce82cc194678d371b2"} Jul 29 01:07:57.824919 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:57.824877 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" event={"ID":"eaaec969-fdff-4ff3-ac0f-a68c83f8a233","Type":"ContainerStarted","Data":"41da1e60dbc171251033adb3584cb49daad95a1db1f6cee70df9073a47e53eff"} Jul 29 01:07:57.846250 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:57.846186 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" podStartSLOduration=1.846169174 podStartE2EDuration="1.846169174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:07:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:07:57.845579427 +0000 UTC m=+1929.504805315" watchObservedRunningTime="2026-07-29 01:07:57.846169174 +0000 UTC m=+1929.505395066" Jul 29 01:07:58.621864 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:58.621830 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:58.626352 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:58.626304 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:58.829367 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:58.829334 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:07:58.830093 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:07:58.830074 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:08:14.262381 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:14.261888 2558 patch_prober.go:28] interesting pod/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": read tcp 10.132.0.2:33746->10.132.0.47:15021: read: connection reset by peer" start-of-body= Jul 29 01:08:14.262381 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:14.261966 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" podUID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": read tcp 10.132.0.2:33746->10.132.0.47:15021: read: connection reset by peer" Jul 29 01:08:14.911037 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:14.911002 2558 generic.go:356] "Generic (PLEG): container finished" podID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerID="41da1e60dbc171251033adb3584cb49daad95a1db1f6cee70df9073a47e53eff" exitCode=0 Jul 29 01:08:14.911250 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:14.911081 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" event={"ID":"eaaec969-fdff-4ff3-ac0f-a68c83f8a233","Type":"ContainerDied","Data":"41da1e60dbc171251033adb3584cb49daad95a1db1f6cee70df9073a47e53eff"} Jul 29 01:08:14.911564 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:14.911548 2558 scope.go:117] "RemoveContainer" containerID="41da1e60dbc171251033adb3584cb49daad95a1db1f6cee70df9073a47e53eff" Jul 29 01:08:14.912668 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:14.912631 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:08:14.912736 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:14.912710 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:08:14.912776 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:14.912756 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:08:15.917985 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:15.917951 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" event={"ID":"eaaec969-fdff-4ff3-ac0f-a68c83f8a233","Type":"ContainerStarted","Data":"0e0a9fa3704946eddcbeffcaaa49697217969c32ac9aa84239b8569795022a3e"} Jul 29 01:08:16.622671 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:16.622629 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:08:16.622875 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:16.622853 2558 patch_prober.go:28] interesting pod/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 29 01:08:16.622931 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:16.622912 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" podUID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 29 01:08:17.622382 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:17.622348 2558 patch_prober.go:28] interesting pod/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 29 01:08:17.622770 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:17.622397 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" podUID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 29 01:08:18.621841 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:18.621811 2558 patch_prober.go:28] interesting pod/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 29 01:08:18.622019 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:18.621861 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" podUID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 29 01:08:19.622466 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:19.622426 2558 patch_prober.go:28] interesting pod/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 29 01:08:19.622961 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:19.622495 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" podUID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 29 01:08:20.625177 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:20.625145 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:08:20.625646 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:20.625433 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:08:20.626055 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:20.626039 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:08:48.846791 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:48.846757 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt"] Jul 29 01:08:48.847279 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:48.846959 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" podUID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerName="istio-proxy" containerID="cri-o://0e0a9fa3704946eddcbeffcaaa49697217969c32ac9aa84239b8569795022a3e" gracePeriod=30 Jul 29 01:08:50.627693 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:50.627651 2558 patch_prober.go:28] interesting pod/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 01:08:50.628062 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:50.627734 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" podUID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 01:08:54.096906 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.096879 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:08:54.104553 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.104528 2558 generic.go:356] "Generic (PLEG): container finished" podID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerID="0e0a9fa3704946eddcbeffcaaa49697217969c32ac9aa84239b8569795022a3e" exitCode=0 Jul 29 01:08:54.104663 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.104573 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" Jul 29 01:08:54.104663 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.104583 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" event={"ID":"eaaec969-fdff-4ff3-ac0f-a68c83f8a233","Type":"ContainerDied","Data":"0e0a9fa3704946eddcbeffcaaa49697217969c32ac9aa84239b8569795022a3e"} Jul 29 01:08:54.104663 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.104643 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt" event={"ID":"eaaec969-fdff-4ff3-ac0f-a68c83f8a233","Type":"ContainerDied","Data":"b5074defee7c1eb166e0eda98f9f3566c850e25bb63c59ce82cc194678d371b2"} Jul 29 01:08:54.104784 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.104675 2558 scope.go:117] "RemoveContainer" containerID="0e0a9fa3704946eddcbeffcaaa49697217969c32ac9aa84239b8569795022a3e" Jul 29 01:08:54.112966 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.112948 2558 scope.go:117] "RemoveContainer" containerID="41da1e60dbc171251033adb3584cb49daad95a1db1f6cee70df9073a47e53eff" Jul 29 01:08:54.124065 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.124035 2558 scope.go:117] "RemoveContainer" containerID="0e0a9fa3704946eddcbeffcaaa49697217969c32ac9aa84239b8569795022a3e" Jul 29 01:08:54.124344 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:08:54.124304 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e0a9fa3704946eddcbeffcaaa49697217969c32ac9aa84239b8569795022a3e\": container with ID starting with 0e0a9fa3704946eddcbeffcaaa49697217969c32ac9aa84239b8569795022a3e not found: ID does not exist" containerID="0e0a9fa3704946eddcbeffcaaa49697217969c32ac9aa84239b8569795022a3e" Jul 29 01:08:54.124399 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.124353 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e0a9fa3704946eddcbeffcaaa49697217969c32ac9aa84239b8569795022a3e"} err="failed to get container status \"0e0a9fa3704946eddcbeffcaaa49697217969c32ac9aa84239b8569795022a3e\": rpc error: code = NotFound desc = could not find container \"0e0a9fa3704946eddcbeffcaaa49697217969c32ac9aa84239b8569795022a3e\": container with ID starting with 0e0a9fa3704946eddcbeffcaaa49697217969c32ac9aa84239b8569795022a3e not found: ID does not exist" Jul 29 01:08:54.124399 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.124370 2558 scope.go:117] "RemoveContainer" containerID="41da1e60dbc171251033adb3584cb49daad95a1db1f6cee70df9073a47e53eff" Jul 29 01:08:54.124652 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:08:54.124634 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41da1e60dbc171251033adb3584cb49daad95a1db1f6cee70df9073a47e53eff\": container with ID starting with 41da1e60dbc171251033adb3584cb49daad95a1db1f6cee70df9073a47e53eff not found: ID does not exist" containerID="41da1e60dbc171251033adb3584cb49daad95a1db1f6cee70df9073a47e53eff" Jul 29 01:08:54.124715 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.124661 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41da1e60dbc171251033adb3584cb49daad95a1db1f6cee70df9073a47e53eff"} err="failed to get container status \"41da1e60dbc171251033adb3584cb49daad95a1db1f6cee70df9073a47e53eff\": rpc error: code = NotFound desc = could not find container \"41da1e60dbc171251033adb3584cb49daad95a1db1f6cee70df9073a47e53eff\": container with ID starting with 41da1e60dbc171251033adb3584cb49daad95a1db1f6cee70df9073a47e53eff not found: ID does not exist" Jul 29 01:08:54.181650 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.181614 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6hdv\" (UniqueName: \"kubernetes.io/projected/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-kube-api-access-c6hdv\") pod \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " Jul 29 01:08:54.181781 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.181671 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-workload-certs\") pod \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " Jul 29 01:08:54.181781 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.181701 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istiod-ca-cert\") pod \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " Jul 29 01:08:54.181781 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.181755 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-workload-socket\") pod \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " Jul 29 01:08:54.181946 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.181799 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-data\") pod \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " Jul 29 01:08:54.181946 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.181837 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-envoy\") pod \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " Jul 29 01:08:54.181946 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.181870 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-credential-socket\") pod \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " Jul 29 01:08:54.181946 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.181899 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-podinfo\") pod \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " Jul 29 01:08:54.182143 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.181945 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-token\") pod \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\" (UID: \"eaaec969-fdff-4ff3-ac0f-a68c83f8a233\") " Jul 29 01:08:54.182143 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.182009 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eaaec969-fdff-4ff3-ac0f-a68c83f8a233" (UID: "eaaec969-fdff-4ff3-ac0f-a68c83f8a233"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:08:54.182143 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.182063 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eaaec969-fdff-4ff3-ac0f-a68c83f8a233" (UID: "eaaec969-fdff-4ff3-ac0f-a68c83f8a233"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:08:54.182143 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.182110 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eaaec969-fdff-4ff3-ac0f-a68c83f8a233" (UID: "eaaec969-fdff-4ff3-ac0f-a68c83f8a233"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:08:54.182383 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.182237 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eaaec969-fdff-4ff3-ac0f-a68c83f8a233" (UID: "eaaec969-fdff-4ff3-ac0f-a68c83f8a233"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:08:54.182471 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.182453 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-workload-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:08:54.182517 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.182476 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-credential-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:08:54.182517 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.182490 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-workload-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:08:54.182517 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.182501 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istiod-ca-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:08:54.182688 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.182666 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-data" (OuterVolumeSpecName: "istio-data") pod "eaaec969-fdff-4ff3-ac0f-a68c83f8a233" (UID: "eaaec969-fdff-4ff3-ac0f-a68c83f8a233"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:08:54.184267 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.184242 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eaaec969-fdff-4ff3-ac0f-a68c83f8a233" (UID: "eaaec969-fdff-4ff3-ac0f-a68c83f8a233"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:08:54.184267 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.184259 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-kube-api-access-c6hdv" (OuterVolumeSpecName: "kube-api-access-c6hdv") pod "eaaec969-fdff-4ff3-ac0f-a68c83f8a233" (UID: "eaaec969-fdff-4ff3-ac0f-a68c83f8a233"). InnerVolumeSpecName "kube-api-access-c6hdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:08:54.184546 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.184531 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eaaec969-fdff-4ff3-ac0f-a68c83f8a233" (UID: "eaaec969-fdff-4ff3-ac0f-a68c83f8a233"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 01:08:54.184624 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.184544 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-token" (OuterVolumeSpecName: "istio-token") pod "eaaec969-fdff-4ff3-ac0f-a68c83f8a233" (UID: "eaaec969-fdff-4ff3-ac0f-a68c83f8a233"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:08:54.283563 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.283539 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-podinfo\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:08:54.283563 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.283561 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:08:54.283695 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.283570 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6hdv\" (UniqueName: \"kubernetes.io/projected/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-kube-api-access-c6hdv\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:08:54.283695 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.283581 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-data\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:08:54.283695 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.283591 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eaaec969-fdff-4ff3-ac0f-a68c83f8a233-istio-envoy\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:08:54.424963 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.424937 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt"] Jul 29 01:08:54.427973 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.427949 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b8aa042b-openshift-default-6f97667568-7rxlt"] Jul 29 01:08:54.789650 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:08:54.789571 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" path="/var/lib/kubelet/pods/eaaec969-fdff-4ff3-ac0f-a68c83f8a233/volumes" Jul 29 01:09:08.029813 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.029776 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6"] Jul 29 01:09:08.030386 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.030367 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerName="istio-proxy" Jul 29 01:09:08.030431 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.030390 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerName="istio-proxy" Jul 29 01:09:08.030507 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.030496 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerName="istio-proxy" Jul 29 01:09:08.030542 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.030518 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f0d64c8-751d-4e55-bdf2-91b47c885ef9" containerName="istio-proxy" Jul 29 01:09:08.035584 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.035559 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.037437 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.037410 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9bb5bd21-openshift-default-dockercfg-d8czd\"" Jul 29 01:09:08.044036 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.044013 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6"] Jul 29 01:09:08.202596 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.202555 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h869w\" (UniqueName: \"kubernetes.io/projected/6bd37fc1-6172-4b45-a62d-912e6ee451a6-kube-api-access-h869w\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.202773 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.202650 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-credential-socket\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.202773 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.202668 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-podinfo\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.202773 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.202692 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-token\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.202773 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.202719 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-envoy\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.202773 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.202755 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-data\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.202966 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.202848 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-workload-socket\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.202966 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.202887 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-workload-certs\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.202966 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.202908 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istiod-ca-cert\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.303413 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.303328 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-envoy\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.303413 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.303374 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-data\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.303636 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.303427 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-workload-socket\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.303636 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.303454 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-workload-certs\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.303636 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.303479 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istiod-ca-cert\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.303636 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.303520 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h869w\" (UniqueName: \"kubernetes.io/projected/6bd37fc1-6172-4b45-a62d-912e6ee451a6-kube-api-access-h869w\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.303636 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.303597 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-credential-socket\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.303636 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.303623 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-podinfo\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.303927 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.303650 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-token\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.303927 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.303839 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-workload-socket\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.303927 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.303863 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-data\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.304082 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.304006 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-credential-socket\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.304167 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.304144 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-workload-certs\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.304373 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.304356 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istiod-ca-cert\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.305672 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.305642 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-envoy\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.305974 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.305955 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-podinfo\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.310794 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.310768 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h869w\" (UniqueName: \"kubernetes.io/projected/6bd37fc1-6172-4b45-a62d-912e6ee451a6-kube-api-access-h869w\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.310881 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.310806 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-token\") pod \"e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.347814 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.347791 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:08.483710 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.483683 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6"] Jul 29 01:09:08.485876 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:09:08.485852 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd37fc1_6172_4b45_a62d_912e6ee451a6.slice/crio-e8a97c92c07e5899e1a3005557031ea1882969e5cfc92b38847811aa214839f5 WatchSource:0}: Error finding container e8a97c92c07e5899e1a3005557031ea1882969e5cfc92b38847811aa214839f5: Status 404 returned error can't find the container with id e8a97c92c07e5899e1a3005557031ea1882969e5cfc92b38847811aa214839f5 Jul 29 01:09:08.488468 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.488435 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:09:08.488574 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.488512 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:09:08.488574 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:08.488556 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:09:09.178545 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:09.178508 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" event={"ID":"6bd37fc1-6172-4b45-a62d-912e6ee451a6","Type":"ContainerStarted","Data":"9ac39195d0c24ecb7c3a2e8c7c0a5d4c822a8ac57156df455d21f74be2ec27e3"} Jul 29 01:09:09.178933 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:09.178567 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" event={"ID":"6bd37fc1-6172-4b45-a62d-912e6ee451a6","Type":"ContainerStarted","Data":"e8a97c92c07e5899e1a3005557031ea1882969e5cfc92b38847811aa214839f5"} Jul 29 01:09:09.197133 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:09.197078 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" podStartSLOduration=1.197060922 podStartE2EDuration="1.197060922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:09:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:09:09.196277674 +0000 UTC m=+2000.855503565" watchObservedRunningTime="2026-07-29 01:09:09.197060922 +0000 UTC m=+2000.856286817" Jul 29 01:09:09.348401 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:09.348368 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:09.353204 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:09.353183 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:10.183007 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:10.182975 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:10.183964 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:10.183945 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:26.224452 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:26.224416 2558 patch_prober.go:28] interesting pod/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 29 01:09:26.226067 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:26.224488 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 29 01:09:27.271158 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:27.271124 2558 generic.go:356] "Generic (PLEG): container finished" podID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerID="9ac39195d0c24ecb7c3a2e8c7c0a5d4c822a8ac57156df455d21f74be2ec27e3" exitCode=0 Jul 29 01:09:27.271565 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:27.271196 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" event={"ID":"6bd37fc1-6172-4b45-a62d-912e6ee451a6","Type":"ContainerDied","Data":"9ac39195d0c24ecb7c3a2e8c7c0a5d4c822a8ac57156df455d21f74be2ec27e3"} Jul 29 01:09:27.271565 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:27.271555 2558 scope.go:117] "RemoveContainer" containerID="9ac39195d0c24ecb7c3a2e8c7c0a5d4c822a8ac57156df455d21f74be2ec27e3" Jul 29 01:09:27.272507 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:27.272478 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:09:27.272621 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:27.272561 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:09:27.272621 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:27.272610 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:09:28.281281 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:28.281231 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" event={"ID":"6bd37fc1-6172-4b45-a62d-912e6ee451a6","Type":"ContainerStarted","Data":"a0d8da2374d221bbffba6a597afc4cd0bcd2afbd23a30d669e7bde835683ffa1"} Jul 29 01:09:28.348297 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:28.348262 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:28.348509 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:28.348483 2558 patch_prober.go:28] interesting pod/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 29 01:09:28.348569 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:28.348549 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 29 01:09:29.348639 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:29.348600 2558 patch_prober.go:28] interesting pod/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 29 01:09:29.349103 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:29.348658 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 29 01:09:30.348987 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:30.348953 2558 patch_prober.go:28] interesting pod/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 29 01:09:30.349389 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:30.349009 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 29 01:09:31.348952 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:31.348914 2558 patch_prober.go:28] interesting pod/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 29 01:09:31.349119 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:31.348965 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 29 01:09:32.348204 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:32.348169 2558 patch_prober.go:28] interesting pod/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 29 01:09:32.348400 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:32.348221 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 29 01:09:33.352166 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:33.352135 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:33.352583 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:33.352474 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:09:33.353089 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:09:33.353073 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:10:48.889652 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:10:48.889572 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:10:48.893809 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:10:48.893788 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:10:48.894096 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:10:48.894080 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 01:10:48.898252 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:10:48.898233 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 01:10:48.962574 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:10:48.962550 2558 scope.go:117] "RemoveContainer" containerID="3f322734237145561fda084ecab603b1d4fcc8091fa16b6903ddfabbb137217d" Jul 29 01:11:17.365468 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:17.365434 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6"] Jul 29 01:11:17.365851 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:17.365691 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerName="istio-proxy" containerID="cri-o://a0d8da2374d221bbffba6a597afc4cd0bcd2afbd23a30d669e7bde835683ffa1" gracePeriod=30 Jul 29 01:11:18.355087 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:18.355051 2558 patch_prober.go:28] interesting pod/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 01:11:18.355263 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:18.355115 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 01:11:22.607212 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.607189 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:11:22.682224 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.682162 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-data\") pod \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " Jul 29 01:11:22.682224 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.682197 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istiod-ca-cert\") pod \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " Jul 29 01:11:22.682409 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.682232 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-workload-certs\") pod \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " Jul 29 01:11:22.682409 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.682261 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h869w\" (UniqueName: \"kubernetes.io/projected/6bd37fc1-6172-4b45-a62d-912e6ee451a6-kube-api-access-h869w\") pod \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " Jul 29 01:11:22.682409 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.682279 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-workload-socket\") pod \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " Jul 29 01:11:22.682409 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.682296 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-token\") pod \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " Jul 29 01:11:22.682409 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.682339 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-podinfo\") pod \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " Jul 29 01:11:22.682409 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.682364 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-credential-socket\") pod \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " Jul 29 01:11:22.682409 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.682404 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-envoy\") pod \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\" (UID: \"6bd37fc1-6172-4b45-a62d-912e6ee451a6\") " Jul 29 01:11:22.682769 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.682639 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6bd37fc1-6172-4b45-a62d-912e6ee451a6" (UID: "6bd37fc1-6172-4b45-a62d-912e6ee451a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:11:22.682769 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.682598 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6bd37fc1-6172-4b45-a62d-912e6ee451a6" (UID: "6bd37fc1-6172-4b45-a62d-912e6ee451a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:11:22.682877 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.682851 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6bd37fc1-6172-4b45-a62d-912e6ee451a6" (UID: "6bd37fc1-6172-4b45-a62d-912e6ee451a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:11:22.682932 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.682860 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6bd37fc1-6172-4b45-a62d-912e6ee451a6" (UID: "6bd37fc1-6172-4b45-a62d-912e6ee451a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:11:22.683073 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.683056 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "6bd37fc1-6172-4b45-a62d-912e6ee451a6" (UID: "6bd37fc1-6172-4b45-a62d-912e6ee451a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:11:22.684723 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.684693 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6bd37fc1-6172-4b45-a62d-912e6ee451a6" (UID: "6bd37fc1-6172-4b45-a62d-912e6ee451a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:11:22.684723 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.684702 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6bd37fc1-6172-4b45-a62d-912e6ee451a6" (UID: "6bd37fc1-6172-4b45-a62d-912e6ee451a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 01:11:22.684953 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.684936 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bd37fc1-6172-4b45-a62d-912e6ee451a6-kube-api-access-h869w" (OuterVolumeSpecName: "kube-api-access-h869w") pod "6bd37fc1-6172-4b45-a62d-912e6ee451a6" (UID: "6bd37fc1-6172-4b45-a62d-912e6ee451a6"). InnerVolumeSpecName "kube-api-access-h869w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:11:22.685016 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.684960 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "6bd37fc1-6172-4b45-a62d-912e6ee451a6" (UID: "6bd37fc1-6172-4b45-a62d-912e6ee451a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:11:22.783682 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.783655 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-data\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:11:22.783682 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.783679 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istiod-ca-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:11:22.783837 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.783693 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-workload-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:11:22.783837 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.783705 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h869w\" (UniqueName: \"kubernetes.io/projected/6bd37fc1-6172-4b45-a62d-912e6ee451a6-kube-api-access-h869w\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:11:22.783837 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.783719 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-workload-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:11:22.783837 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.783731 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:11:22.783837 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.783743 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-podinfo\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:11:22.783837 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.783755 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-credential-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:11:22.783837 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.783767 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bd37fc1-6172-4b45-a62d-912e6ee451a6-istio-envoy\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:11:22.829756 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.829733 2558 generic.go:356] "Generic (PLEG): container finished" podID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerID="a0d8da2374d221bbffba6a597afc4cd0bcd2afbd23a30d669e7bde835683ffa1" exitCode=0 Jul 29 01:11:22.829869 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.829815 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" event={"ID":"6bd37fc1-6172-4b45-a62d-912e6ee451a6","Type":"ContainerDied","Data":"a0d8da2374d221bbffba6a597afc4cd0bcd2afbd23a30d669e7bde835683ffa1"} Jul 29 01:11:22.829869 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.829829 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" Jul 29 01:11:22.829984 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.829872 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6" event={"ID":"6bd37fc1-6172-4b45-a62d-912e6ee451a6","Type":"ContainerDied","Data":"e8a97c92c07e5899e1a3005557031ea1882969e5cfc92b38847811aa214839f5"} Jul 29 01:11:22.829984 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.829897 2558 scope.go:117] "RemoveContainer" containerID="a0d8da2374d221bbffba6a597afc4cd0bcd2afbd23a30d669e7bde835683ffa1" Jul 29 01:11:22.838547 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.838530 2558 scope.go:117] "RemoveContainer" containerID="9ac39195d0c24ecb7c3a2e8c7c0a5d4c822a8ac57156df455d21f74be2ec27e3" Jul 29 01:11:22.845786 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.845761 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6"] Jul 29 01:11:22.847391 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.847372 2558 scope.go:117] "RemoveContainer" containerID="a0d8da2374d221bbffba6a597afc4cd0bcd2afbd23a30d669e7bde835683ffa1" Jul 29 01:11:22.847659 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:11:22.847641 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0d8da2374d221bbffba6a597afc4cd0bcd2afbd23a30d669e7bde835683ffa1\": container with ID starting with a0d8da2374d221bbffba6a597afc4cd0bcd2afbd23a30d669e7bde835683ffa1 not found: ID does not exist" containerID="a0d8da2374d221bbffba6a597afc4cd0bcd2afbd23a30d669e7bde835683ffa1" Jul 29 01:11:22.847711 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.847665 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0d8da2374d221bbffba6a597afc4cd0bcd2afbd23a30d669e7bde835683ffa1"} err="failed to get container status \"a0d8da2374d221bbffba6a597afc4cd0bcd2afbd23a30d669e7bde835683ffa1\": rpc error: code = NotFound desc = could not find container \"a0d8da2374d221bbffba6a597afc4cd0bcd2afbd23a30d669e7bde835683ffa1\": container with ID starting with a0d8da2374d221bbffba6a597afc4cd0bcd2afbd23a30d669e7bde835683ffa1 not found: ID does not exist" Jul 29 01:11:22.847711 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.847682 2558 scope.go:117] "RemoveContainer" containerID="9ac39195d0c24ecb7c3a2e8c7c0a5d4c822a8ac57156df455d21f74be2ec27e3" Jul 29 01:11:22.847919 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:11:22.847904 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ac39195d0c24ecb7c3a2e8c7c0a5d4c822a8ac57156df455d21f74be2ec27e3\": container with ID starting with 9ac39195d0c24ecb7c3a2e8c7c0a5d4c822a8ac57156df455d21f74be2ec27e3 not found: ID does not exist" containerID="9ac39195d0c24ecb7c3a2e8c7c0a5d4c822a8ac57156df455d21f74be2ec27e3" Jul 29 01:11:22.847973 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.847924 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ac39195d0c24ecb7c3a2e8c7c0a5d4c822a8ac57156df455d21f74be2ec27e3"} err="failed to get container status \"9ac39195d0c24ecb7c3a2e8c7c0a5d4c822a8ac57156df455d21f74be2ec27e3\": rpc error: code = NotFound desc = could not find container \"9ac39195d0c24ecb7c3a2e8c7c0a5d4c822a8ac57156df455d21f74be2ec27e3\": container with ID starting with 9ac39195d0c24ecb7c3a2e8c7c0a5d4c822a8ac57156df455d21f74be2ec27e3 not found: ID does not exist" Jul 29 01:11:22.848186 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:22.848170 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9bb5bd21-openshift-default-6656695d68-4grm6"] Jul 29 01:11:24.788978 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:11:24.788946 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" path="/var/lib/kubelet/pods/6bd37fc1-6172-4b45-a62d-912e6ee451a6/volumes" Jul 29 01:12:12.328337 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.328288 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4"] Jul 29 01:12:12.328738 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.328673 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerName="istio-proxy" Jul 29 01:12:12.328738 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.328686 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerName="istio-proxy" Jul 29 01:12:12.328738 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.328703 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerName="istio-proxy" Jul 29 01:12:12.328738 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.328708 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerName="istio-proxy" Jul 29 01:12:12.328872 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.328799 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerName="istio-proxy" Jul 29 01:12:12.328872 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.328810 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaaec969-fdff-4ff3-ac0f-a68c83f8a233" containerName="istio-proxy" Jul 29 01:12:12.328872 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.328819 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerName="istio-proxy" Jul 29 01:12:12.331777 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.331759 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.333441 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.333418 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-dda27e-openshift-default-dockercfg-rjxq9\"" Jul 29 01:12:12.341035 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.341014 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4"] Jul 29 01:12:12.407983 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.407949 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-workload-socket\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.407983 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.407984 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61906f63-5663-42d0-adc8-abd2dba31083-istiod-ca-cert\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.408213 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.408010 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-credential-socket\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.408213 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.408114 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-istio-data\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.408213 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.408151 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-istio-envoy\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.408213 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.408197 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61906f63-5663-42d0-adc8-abd2dba31083-istio-podinfo\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.408458 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.408227 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61906f63-5663-42d0-adc8-abd2dba31083-istio-token\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.408458 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.408281 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-workload-certs\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.408458 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.408333 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll9gg\" (UniqueName: \"kubernetes.io/projected/61906f63-5663-42d0-adc8-abd2dba31083-kube-api-access-ll9gg\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.509220 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.509180 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-workload-socket\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.509445 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.509234 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61906f63-5663-42d0-adc8-abd2dba31083-istiod-ca-cert\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.509445 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.509290 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-credential-socket\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.509445 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.509354 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-istio-data\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.509445 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.509388 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-istio-envoy\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.509445 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.509416 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61906f63-5663-42d0-adc8-abd2dba31083-istio-podinfo\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.509445 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.509443 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61906f63-5663-42d0-adc8-abd2dba31083-istio-token\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.509768 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.509494 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-workload-certs\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.509768 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.509518 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll9gg\" (UniqueName: \"kubernetes.io/projected/61906f63-5663-42d0-adc8-abd2dba31083-kube-api-access-ll9gg\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.509768 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.509686 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-credential-socket\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.509768 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.509739 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-workload-socket\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.509985 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.509786 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-istio-data\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.510045 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.509994 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-workload-certs\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.510281 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.510244 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61906f63-5663-42d0-adc8-abd2dba31083-istiod-ca-cert\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.511996 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.511962 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-istio-envoy\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.512404 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.512385 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61906f63-5663-42d0-adc8-abd2dba31083-istio-podinfo\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.519737 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.519706 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61906f63-5663-42d0-adc8-abd2dba31083-istio-token\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.519846 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.519822 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll9gg\" (UniqueName: \"kubernetes.io/projected/61906f63-5663-42d0-adc8-abd2dba31083-kube-api-access-ll9gg\") pod \"e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.643329 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.643216 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:12.782176 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.782150 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4"] Jul 29 01:12:12.784883 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:12:12.784858 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61906f63_5663_42d0_adc8_abd2dba31083.slice/crio-801db5c38100d3876bc5e51064d0c9751d189bc17611328732a0b540edeb0c21 WatchSource:0}: Error finding container 801db5c38100d3876bc5e51064d0c9751d189bc17611328732a0b540edeb0c21: Status 404 returned error can't find the container with id 801db5c38100d3876bc5e51064d0c9751d189bc17611328732a0b540edeb0c21 Jul 29 01:12:12.787044 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.787007 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:12:12.787131 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.787078 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:12:12.787131 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:12.787114 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:12:13.063580 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:13.063547 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" event={"ID":"61906f63-5663-42d0-adc8-abd2dba31083","Type":"ContainerStarted","Data":"93541f2dd9107681b128ba3fbef7ef2ab7900f5466cfd8156b99d92524f128d5"} Jul 29 01:12:13.063743 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:13.063597 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" event={"ID":"61906f63-5663-42d0-adc8-abd2dba31083","Type":"ContainerStarted","Data":"801db5c38100d3876bc5e51064d0c9751d189bc17611328732a0b540edeb0c21"} Jul 29 01:12:13.082138 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:13.082085 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" podStartSLOduration=1.082070251 podStartE2EDuration="1.082070251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:12:13.08087885 +0000 UTC m=+2184.740104743" watchObservedRunningTime="2026-07-29 01:12:13.082070251 +0000 UTC m=+2184.741296143" Jul 29 01:12:13.643912 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:13.643879 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:13.649056 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:13.649030 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:14.068541 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:14.068514 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:14.069492 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:14.069475 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:29.563471 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:29.563016 2558 patch_prober.go:28] interesting pod/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": read tcp 10.132.0.2:55420->10.132.0.49:15021: read: connection reset by peer" start-of-body= Jul 29 01:12:29.563471 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:29.563102 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" podUID="61906f63-5663-42d0-adc8-abd2dba31083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": read tcp 10.132.0.2:55420->10.132.0.49:15021: read: connection reset by peer" Jul 29 01:12:30.142957 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:30.142924 2558 generic.go:356] "Generic (PLEG): container finished" podID="61906f63-5663-42d0-adc8-abd2dba31083" containerID="93541f2dd9107681b128ba3fbef7ef2ab7900f5466cfd8156b99d92524f128d5" exitCode=0 Jul 29 01:12:30.143133 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:30.142996 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" event={"ID":"61906f63-5663-42d0-adc8-abd2dba31083","Type":"ContainerDied","Data":"93541f2dd9107681b128ba3fbef7ef2ab7900f5466cfd8156b99d92524f128d5"} Jul 29 01:12:30.143434 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:30.143416 2558 scope.go:117] "RemoveContainer" containerID="93541f2dd9107681b128ba3fbef7ef2ab7900f5466cfd8156b99d92524f128d5" Jul 29 01:12:30.144386 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:30.144358 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:12:30.144441 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:30.144425 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:12:30.144477 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:30.144459 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:12:31.149660 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:31.149623 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" event={"ID":"61906f63-5663-42d0-adc8-abd2dba31083","Type":"ContainerStarted","Data":"26fb0f3d2ba649645913a7f323e06d0a886543e8579e17c3471a0ea95a0dbf98"} Jul 29 01:12:31.644226 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:31.644188 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:31.644407 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:31.644387 2558 patch_prober.go:28] interesting pod/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 29 01:12:31.644456 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:31.644435 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" podUID="61906f63-5663-42d0-adc8-abd2dba31083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 29 01:12:32.643629 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:32.643593 2558 patch_prober.go:28] interesting pod/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 29 01:12:32.644032 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:32.643645 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" podUID="61906f63-5663-42d0-adc8-abd2dba31083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 29 01:12:33.643827 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:33.643790 2558 patch_prober.go:28] interesting pod/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 29 01:12:33.644212 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:33.643842 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" podUID="61906f63-5663-42d0-adc8-abd2dba31083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 29 01:12:34.643719 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:34.643681 2558 patch_prober.go:28] interesting pod/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 29 01:12:34.643918 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:34.643737 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" podUID="61906f63-5663-42d0-adc8-abd2dba31083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 29 01:12:35.647446 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:35.647415 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:35.647820 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:35.647663 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:35.648292 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:35.648272 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:12:40.547887 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.547845 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8"] Jul 29 01:12:40.548535 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.548512 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerName="istio-proxy" Jul 29 01:12:40.548621 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.548538 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bd37fc1-6172-4b45-a62d-912e6ee451a6" containerName="istio-proxy" Jul 29 01:12:40.551403 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.551380 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.553416 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.553389 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-a1744c-openshift-default-dockercfg-f2v5d\"" Jul 29 01:12:40.562267 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.562245 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8"] Jul 29 01:12:40.667724 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.667686 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-credential-socket\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.667899 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.667747 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-podinfo\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.667899 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.667787 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-token\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.667899 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.667869 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-workload-socket\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.668037 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.667939 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istiod-ca-cert\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.668037 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.667974 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-data\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.668111 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.668029 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q92km\" (UniqueName: \"kubernetes.io/projected/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-kube-api-access-q92km\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.668111 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.668088 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-envoy\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.668199 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.668119 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-workload-certs\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.769697 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.769664 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-envoy\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.769867 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.769717 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-workload-certs\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.769867 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.769762 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-credential-socket\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.769867 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.769802 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-podinfo\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.769867 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.769841 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-token\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.770087 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.769872 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-workload-socket\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.770087 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.769919 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istiod-ca-cert\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.770087 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.769948 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-data\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.770087 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.769978 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q92km\" (UniqueName: \"kubernetes.io/projected/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-kube-api-access-q92km\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.770304 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.770204 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-credential-socket\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.770450 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.770398 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-workload-certs\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.770518 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.770451 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-data\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.770610 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.770584 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-workload-socket\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.770756 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.770738 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istiod-ca-cert\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.771998 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.771978 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-envoy\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.772273 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.772254 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-podinfo\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.779574 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.779550 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-token\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.779964 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.779944 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q92km\" (UniqueName: \"kubernetes.io/projected/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-kube-api-access-q92km\") pod \"e2e-inf-b-a1744c-openshift-default-77bd49876-795q8\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:40.864841 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:40.864792 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:41.002325 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:41.002281 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8"] Jul 29 01:12:41.005670 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:12:41.005644 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4140611d_c2ac_4b7e_84e0_3acd80bd0b88.slice/crio-ad738aa512999a92cf64c063e1151462f4caa861c4e4292f7859784dbf85dbb7 WatchSource:0}: Error finding container ad738aa512999a92cf64c063e1151462f4caa861c4e4292f7859784dbf85dbb7: Status 404 returned error can't find the container with id ad738aa512999a92cf64c063e1151462f4caa861c4e4292f7859784dbf85dbb7 Jul 29 01:12:41.007967 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:41.007935 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:12:41.008059 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:41.008000 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:12:41.008059 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:41.008027 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:12:41.203495 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:41.203396 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" event={"ID":"4140611d-c2ac-4b7e-84e0-3acd80bd0b88","Type":"ContainerStarted","Data":"40f1e20fd7f808f6b703da7eb6349de74d280d481490d230a8121c6356b31309"} Jul 29 01:12:41.203495 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:41.203453 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" event={"ID":"4140611d-c2ac-4b7e-84e0-3acd80bd0b88","Type":"ContainerStarted","Data":"ad738aa512999a92cf64c063e1151462f4caa861c4e4292f7859784dbf85dbb7"} Jul 29 01:12:41.222164 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:41.222106 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" podStartSLOduration=1.222086472 podStartE2EDuration="1.222086472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:12:41.220088044 +0000 UTC m=+2212.879313937" watchObservedRunningTime="2026-07-29 01:12:41.222086472 +0000 UTC m=+2212.881312366" Jul 29 01:12:41.865756 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:41.865721 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:42.870641 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:42.870609 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:43.213942 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:43.213859 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:43.214796 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:43.214769 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:12:58.884969 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:58.884739 2558 patch_prober.go:28] interesting pod/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": read tcp 10.132.0.2:50924->10.132.0.50:15021: read: connection reset by peer" start-of-body= Jul 29 01:12:58.884969 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:58.884801 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" podUID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": read tcp 10.132.0.2:50924->10.132.0.50:15021: read: connection reset by peer" Jul 29 01:12:59.299475 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:59.299383 2558 generic.go:356] "Generic (PLEG): container finished" podID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerID="40f1e20fd7f808f6b703da7eb6349de74d280d481490d230a8121c6356b31309" exitCode=0 Jul 29 01:12:59.299475 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:59.299457 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" event={"ID":"4140611d-c2ac-4b7e-84e0-3acd80bd0b88","Type":"ContainerDied","Data":"40f1e20fd7f808f6b703da7eb6349de74d280d481490d230a8121c6356b31309"} Jul 29 01:12:59.299849 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:59.299836 2558 scope.go:117] "RemoveContainer" containerID="40f1e20fd7f808f6b703da7eb6349de74d280d481490d230a8121c6356b31309" Jul 29 01:12:59.300905 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:59.300875 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:12:59.300955 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:59.300944 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:12:59.301001 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:12:59.300980 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:13:00.306302 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:13:00.306267 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" event={"ID":"4140611d-c2ac-4b7e-84e0-3acd80bd0b88","Type":"ContainerStarted","Data":"179fd26ffe90843c7fd393b5d9a6ea06f5dcf7235a6257753aaadcf45ec33c98"} Jul 29 01:13:00.865784 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:13:00.865742 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:13:00.865991 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:13:00.865870 2558 patch_prober.go:28] interesting pod/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 29 01:13:00.865991 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:13:00.865930 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" podUID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 29 01:13:01.865668 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:13:01.865620 2558 patch_prober.go:28] interesting pod/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 29 01:13:01.866181 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:13:01.865689 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" podUID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 29 01:13:02.866141 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:13:02.866101 2558 patch_prober.go:28] interesting pod/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 29 01:13:02.866732 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:13:02.866689 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" podUID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 29 01:13:03.866230 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:13:03.866191 2558 patch_prober.go:28] interesting pod/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 29 01:13:03.866797 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:13:03.866763 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" podUID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 29 01:13:04.868598 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:13:04.868568 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:13:04.869006 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:13:04.868936 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:13:04.869561 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:13:04.869544 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:14:46.368768 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:46.368684 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4"] Jul 29 01:14:46.369239 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:46.368919 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" podUID="61906f63-5663-42d0-adc8-abd2dba31083" containerName="istio-proxy" containerID="cri-o://26fb0f3d2ba649645913a7f323e06d0a886543e8579e17c3471a0ea95a0dbf98" gracePeriod=30 Jul 29 01:14:50.649759 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:50.649718 2558 patch_prober.go:28] interesting pod/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 01:14:50.650131 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:50.649775 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" podUID="61906f63-5663-42d0-adc8-abd2dba31083" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 01:14:51.633501 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.633437 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:14:51.742404 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.742372 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-istio-envoy\") pod \"61906f63-5663-42d0-adc8-abd2dba31083\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " Jul 29 01:14:51.742869 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.742416 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-istio-data\") pod \"61906f63-5663-42d0-adc8-abd2dba31083\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " Jul 29 01:14:51.742869 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.742459 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-workload-socket\") pod \"61906f63-5663-42d0-adc8-abd2dba31083\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " Jul 29 01:14:51.742869 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.742482 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-credential-socket\") pod \"61906f63-5663-42d0-adc8-abd2dba31083\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " Jul 29 01:14:51.742869 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.742542 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61906f63-5663-42d0-adc8-abd2dba31083-istiod-ca-cert\") pod \"61906f63-5663-42d0-adc8-abd2dba31083\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " Jul 29 01:14:51.742869 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.742573 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61906f63-5663-42d0-adc8-abd2dba31083-istio-token\") pod \"61906f63-5663-42d0-adc8-abd2dba31083\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " Jul 29 01:14:51.742869 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.742645 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-workload-certs\") pod \"61906f63-5663-42d0-adc8-abd2dba31083\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " Jul 29 01:14:51.742869 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.742671 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ll9gg\" (UniqueName: \"kubernetes.io/projected/61906f63-5663-42d0-adc8-abd2dba31083-kube-api-access-ll9gg\") pod \"61906f63-5663-42d0-adc8-abd2dba31083\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " Jul 29 01:14:51.742869 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.742731 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61906f63-5663-42d0-adc8-abd2dba31083-istio-podinfo\") pod \"61906f63-5663-42d0-adc8-abd2dba31083\" (UID: \"61906f63-5663-42d0-adc8-abd2dba31083\") " Jul 29 01:14:51.742869 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.742804 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "61906f63-5663-42d0-adc8-abd2dba31083" (UID: "61906f63-5663-42d0-adc8-abd2dba31083"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:14:51.742869 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.742817 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "61906f63-5663-42d0-adc8-abd2dba31083" (UID: "61906f63-5663-42d0-adc8-abd2dba31083"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:14:51.743557 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.742988 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61906f63-5663-42d0-adc8-abd2dba31083-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "61906f63-5663-42d0-adc8-abd2dba31083" (UID: "61906f63-5663-42d0-adc8-abd2dba31083"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:14:51.743557 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.743214 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-workload-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:14:51.743557 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.743214 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "61906f63-5663-42d0-adc8-abd2dba31083" (UID: "61906f63-5663-42d0-adc8-abd2dba31083"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:14:51.743557 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.743234 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-credential-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:14:51.743557 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.743250 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61906f63-5663-42d0-adc8-abd2dba31083-istiod-ca-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:14:51.743557 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.743478 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-istio-data" (OuterVolumeSpecName: "istio-data") pod "61906f63-5663-42d0-adc8-abd2dba31083" (UID: "61906f63-5663-42d0-adc8-abd2dba31083"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:14:51.744671 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.744641 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "61906f63-5663-42d0-adc8-abd2dba31083" (UID: "61906f63-5663-42d0-adc8-abd2dba31083"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:14:51.745026 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.745001 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61906f63-5663-42d0-adc8-abd2dba31083-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "61906f63-5663-42d0-adc8-abd2dba31083" (UID: "61906f63-5663-42d0-adc8-abd2dba31083"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 01:14:51.745336 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.745294 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61906f63-5663-42d0-adc8-abd2dba31083-kube-api-access-ll9gg" (OuterVolumeSpecName: "kube-api-access-ll9gg") pod "61906f63-5663-42d0-adc8-abd2dba31083" (UID: "61906f63-5663-42d0-adc8-abd2dba31083"). InnerVolumeSpecName "kube-api-access-ll9gg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:14:51.745407 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.745332 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61906f63-5663-42d0-adc8-abd2dba31083-istio-token" (OuterVolumeSpecName: "istio-token") pod "61906f63-5663-42d0-adc8-abd2dba31083" (UID: "61906f63-5663-42d0-adc8-abd2dba31083"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:14:51.837833 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.837796 2558 generic.go:356] "Generic (PLEG): container finished" podID="61906f63-5663-42d0-adc8-abd2dba31083" containerID="26fb0f3d2ba649645913a7f323e06d0a886543e8579e17c3471a0ea95a0dbf98" exitCode=0 Jul 29 01:14:51.838027 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.837850 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" Jul 29 01:14:51.838027 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.837876 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" event={"ID":"61906f63-5663-42d0-adc8-abd2dba31083","Type":"ContainerDied","Data":"26fb0f3d2ba649645913a7f323e06d0a886543e8579e17c3471a0ea95a0dbf98"} Jul 29 01:14:51.838027 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.837921 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4" event={"ID":"61906f63-5663-42d0-adc8-abd2dba31083","Type":"ContainerDied","Data":"801db5c38100d3876bc5e51064d0c9751d189bc17611328732a0b540edeb0c21"} Jul 29 01:14:51.838027 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.837944 2558 scope.go:117] "RemoveContainer" containerID="26fb0f3d2ba649645913a7f323e06d0a886543e8579e17c3471a0ea95a0dbf98" Jul 29 01:14:51.844367 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.844344 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61906f63-5663-42d0-adc8-abd2dba31083-istio-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:14:51.844367 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.844365 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-workload-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:14:51.844561 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.844373 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ll9gg\" (UniqueName: \"kubernetes.io/projected/61906f63-5663-42d0-adc8-abd2dba31083-kube-api-access-ll9gg\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:14:51.844561 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.844382 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61906f63-5663-42d0-adc8-abd2dba31083-istio-podinfo\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:14:51.844561 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.844391 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-istio-envoy\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:14:51.844561 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.844398 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61906f63-5663-42d0-adc8-abd2dba31083-istio-data\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:14:51.849699 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.849676 2558 scope.go:117] "RemoveContainer" containerID="93541f2dd9107681b128ba3fbef7ef2ab7900f5466cfd8156b99d92524f128d5" Jul 29 01:14:51.858059 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.858033 2558 scope.go:117] "RemoveContainer" containerID="26fb0f3d2ba649645913a7f323e06d0a886543e8579e17c3471a0ea95a0dbf98" Jul 29 01:14:51.858408 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:14:51.858379 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26fb0f3d2ba649645913a7f323e06d0a886543e8579e17c3471a0ea95a0dbf98\": container with ID starting with 26fb0f3d2ba649645913a7f323e06d0a886543e8579e17c3471a0ea95a0dbf98 not found: ID does not exist" containerID="26fb0f3d2ba649645913a7f323e06d0a886543e8579e17c3471a0ea95a0dbf98" Jul 29 01:14:51.858481 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.858420 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26fb0f3d2ba649645913a7f323e06d0a886543e8579e17c3471a0ea95a0dbf98"} err="failed to get container status \"26fb0f3d2ba649645913a7f323e06d0a886543e8579e17c3471a0ea95a0dbf98\": rpc error: code = NotFound desc = could not find container \"26fb0f3d2ba649645913a7f323e06d0a886543e8579e17c3471a0ea95a0dbf98\": container with ID starting with 26fb0f3d2ba649645913a7f323e06d0a886543e8579e17c3471a0ea95a0dbf98 not found: ID does not exist" Jul 29 01:14:51.858481 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.858443 2558 scope.go:117] "RemoveContainer" containerID="93541f2dd9107681b128ba3fbef7ef2ab7900f5466cfd8156b99d92524f128d5" Jul 29 01:14:51.858746 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:14:51.858725 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93541f2dd9107681b128ba3fbef7ef2ab7900f5466cfd8156b99d92524f128d5\": container with ID starting with 93541f2dd9107681b128ba3fbef7ef2ab7900f5466cfd8156b99d92524f128d5 not found: ID does not exist" containerID="93541f2dd9107681b128ba3fbef7ef2ab7900f5466cfd8156b99d92524f128d5" Jul 29 01:14:51.858804 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.858752 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93541f2dd9107681b128ba3fbef7ef2ab7900f5466cfd8156b99d92524f128d5"} err="failed to get container status \"93541f2dd9107681b128ba3fbef7ef2ab7900f5466cfd8156b99d92524f128d5\": rpc error: code = NotFound desc = could not find container \"93541f2dd9107681b128ba3fbef7ef2ab7900f5466cfd8156b99d92524f128d5\": container with ID starting with 93541f2dd9107681b128ba3fbef7ef2ab7900f5466cfd8156b99d92524f128d5 not found: ID does not exist" Jul 29 01:14:51.860047 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.860028 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4"] Jul 29 01:14:51.863220 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:51.863195 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-dda27e-openshift-default-6b68ffb85b-tp9q4"] Jul 29 01:14:52.790166 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:14:52.790128 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61906f63-5663-42d0-adc8-abd2dba31083" path="/var/lib/kubelet/pods/61906f63-5663-42d0-adc8-abd2dba31083/volumes" Jul 29 01:15:03.786343 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:03.786292 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8"] Jul 29 01:15:03.786812 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:03.786508 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" podUID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerName="istio-proxy" containerID="cri-o://179fd26ffe90843c7fd393b5d9a6ea06f5dcf7235a6257753aaadcf45ec33c98" gracePeriod=30 Jul 29 01:15:04.657935 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.657904 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx"] Jul 29 01:15:04.658329 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.658296 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61906f63-5663-42d0-adc8-abd2dba31083" containerName="istio-proxy" Jul 29 01:15:04.658393 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.658329 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="61906f63-5663-42d0-adc8-abd2dba31083" containerName="istio-proxy" Jul 29 01:15:04.658393 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.658358 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61906f63-5663-42d0-adc8-abd2dba31083" containerName="istio-proxy" Jul 29 01:15:04.658393 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.658366 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="61906f63-5663-42d0-adc8-abd2dba31083" containerName="istio-proxy" Jul 29 01:15:04.658501 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.658487 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="61906f63-5663-42d0-adc8-abd2dba31083" containerName="istio-proxy" Jul 29 01:15:04.661591 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.661571 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.664019 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.663999 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-4650f4-openshift-default-dockercfg-8bvlx\"" Jul 29 01:15:04.674928 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.674902 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx"] Jul 29 01:15:04.759871 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.759832 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-credential-socket\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.760044 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.759879 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-token\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.760044 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.759940 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-workload-certs\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.760044 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.759968 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrbr8\" (UniqueName: \"kubernetes.io/projected/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-kube-api-access-hrbr8\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.760044 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.760006 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-workload-socket\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.760205 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.760118 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-envoy\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.760205 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.760155 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-podinfo\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.760276 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.760237 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-data\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.760331 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.760273 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istiod-ca-cert\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.861046 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.861009 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-envoy\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.861046 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.861042 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-podinfo\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.861529 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.861137 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-data\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.861529 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.861176 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istiod-ca-cert\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.861529 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.861246 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-credential-socket\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.861529 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.861273 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-token\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.861529 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.861332 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-workload-certs\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.861529 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.861362 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrbr8\" (UniqueName: \"kubernetes.io/projected/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-kube-api-access-hrbr8\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.861529 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.861399 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-workload-socket\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.861911 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.861607 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-credential-socket\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.861911 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.861722 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-workload-socket\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.861911 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.861858 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-workload-certs\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.862072 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.861948 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-data\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.862072 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.862045 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istiod-ca-cert\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.863671 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.863639 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-podinfo\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.863671 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.863648 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-envoy\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.868675 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.868657 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-token\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.868913 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.868891 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrbr8\" (UniqueName: \"kubernetes.io/projected/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-kube-api-access-hrbr8\") pod \"e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:04.871119 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.871085 2558 patch_prober.go:28] interesting pod/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 01:15:04.871241 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.871161 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" podUID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 01:15:04.972110 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:04.972046 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:05.110322 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:05.110281 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx"] Jul 29 01:15:05.112608 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:15:05.112582 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e37be6_ed6d_4a8a_ab67_aa4005c0e05b.slice/crio-69cf755c267b472db8befa127b1fd027de5d68df029304a0f40334765ff71281 WatchSource:0}: Error finding container 69cf755c267b472db8befa127b1fd027de5d68df029304a0f40334765ff71281: Status 404 returned error can't find the container with id 69cf755c267b472db8befa127b1fd027de5d68df029304a0f40334765ff71281 Jul 29 01:15:05.114783 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:05.114749 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:15:05.114865 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:05.114815 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:15:05.114865 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:05.114843 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:15:05.911594 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:05.911561 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" event={"ID":"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b","Type":"ContainerStarted","Data":"f2294ebd77cb72af2297eeea5ed7a820306af6ed4bf23185249d59d9c2fe96cb"} Jul 29 01:15:05.911977 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:05.911613 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" event={"ID":"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b","Type":"ContainerStarted","Data":"69cf755c267b472db8befa127b1fd027de5d68df029304a0f40334765ff71281"} Jul 29 01:15:05.930222 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:05.930175 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" podStartSLOduration=1.930156031 podStartE2EDuration="1.930156031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:15:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:15:05.928169602 +0000 UTC m=+2357.587395494" watchObservedRunningTime="2026-07-29 01:15:05.930156031 +0000 UTC m=+2357.589381925" Jul 29 01:15:05.972855 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:05.972832 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:06.977617 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:06.977582 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:07.921359 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:07.921327 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:07.922360 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:07.922333 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:08.929405 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:08.929367 2558 generic.go:356] "Generic (PLEG): container finished" podID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerID="179fd26ffe90843c7fd393b5d9a6ea06f5dcf7235a6257753aaadcf45ec33c98" exitCode=0 Jul 29 01:15:08.929829 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:08.929435 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" event={"ID":"4140611d-c2ac-4b7e-84e0-3acd80bd0b88","Type":"ContainerDied","Data":"179fd26ffe90843c7fd393b5d9a6ea06f5dcf7235a6257753aaadcf45ec33c98"} Jul 29 01:15:08.929829 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:08.929514 2558 scope.go:117] "RemoveContainer" containerID="40f1e20fd7f808f6b703da7eb6349de74d280d481490d230a8121c6356b31309" Jul 29 01:15:09.051024 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.050966 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:15:09.102365 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.102338 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-credential-socket\") pod \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " Jul 29 01:15:09.102547 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.102389 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q92km\" (UniqueName: \"kubernetes.io/projected/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-kube-api-access-q92km\") pod \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " Jul 29 01:15:09.102547 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.102413 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-workload-certs\") pod \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " Jul 29 01:15:09.102547 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.102461 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-podinfo\") pod \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " Jul 29 01:15:09.102547 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.102478 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-envoy\") pod \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " Jul 29 01:15:09.102547 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.102495 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-data\") pod \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " Jul 29 01:15:09.102547 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.102538 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-token\") pod \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " Jul 29 01:15:09.102853 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.102555 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-workload-socket\") pod \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " Jul 29 01:15:09.102853 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.102581 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istiod-ca-cert\") pod \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\" (UID: \"4140611d-c2ac-4b7e-84e0-3acd80bd0b88\") " Jul 29 01:15:09.102853 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.102631 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4140611d-c2ac-4b7e-84e0-3acd80bd0b88" (UID: "4140611d-c2ac-4b7e-84e0-3acd80bd0b88"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:15:09.103384 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.102821 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4140611d-c2ac-4b7e-84e0-3acd80bd0b88" (UID: "4140611d-c2ac-4b7e-84e0-3acd80bd0b88"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:15:09.103384 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.102950 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4140611d-c2ac-4b7e-84e0-3acd80bd0b88" (UID: "4140611d-c2ac-4b7e-84e0-3acd80bd0b88"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:15:09.103384 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.103050 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4140611d-c2ac-4b7e-84e0-3acd80bd0b88" (UID: "4140611d-c2ac-4b7e-84e0-3acd80bd0b88"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:15:09.103532 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.103422 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-credential-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:15:09.103532 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.103445 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-workload-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:15:09.103532 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.103459 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istiod-ca-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:15:09.103757 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.103730 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-data" (OuterVolumeSpecName: "istio-data") pod "4140611d-c2ac-4b7e-84e0-3acd80bd0b88" (UID: "4140611d-c2ac-4b7e-84e0-3acd80bd0b88"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:15:09.104949 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.104925 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4140611d-c2ac-4b7e-84e0-3acd80bd0b88" (UID: "4140611d-c2ac-4b7e-84e0-3acd80bd0b88"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 01:15:09.105173 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.105158 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-kube-api-access-q92km" (OuterVolumeSpecName: "kube-api-access-q92km") pod "4140611d-c2ac-4b7e-84e0-3acd80bd0b88" (UID: "4140611d-c2ac-4b7e-84e0-3acd80bd0b88"). InnerVolumeSpecName "kube-api-access-q92km". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:15:09.105585 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.105570 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-token" (OuterVolumeSpecName: "istio-token") pod "4140611d-c2ac-4b7e-84e0-3acd80bd0b88" (UID: "4140611d-c2ac-4b7e-84e0-3acd80bd0b88"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:15:09.105772 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.105760 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4140611d-c2ac-4b7e-84e0-3acd80bd0b88" (UID: "4140611d-c2ac-4b7e-84e0-3acd80bd0b88"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:15:09.204354 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.204279 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q92km\" (UniqueName: \"kubernetes.io/projected/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-kube-api-access-q92km\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:15:09.204354 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.204348 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-podinfo\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:15:09.204354 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.204358 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-envoy\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:15:09.204354 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.204367 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-data\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:15:09.204354 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.204375 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-istio-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:15:09.204664 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.204385 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4140611d-c2ac-4b7e-84e0-3acd80bd0b88-workload-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:15:09.935728 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.935689 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" event={"ID":"4140611d-c2ac-4b7e-84e0-3acd80bd0b88","Type":"ContainerDied","Data":"ad738aa512999a92cf64c063e1151462f4caa861c4e4292f7859784dbf85dbb7"} Jul 29 01:15:09.935728 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.935709 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8" Jul 29 01:15:09.936239 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.935749 2558 scope.go:117] "RemoveContainer" containerID="179fd26ffe90843c7fd393b5d9a6ea06f5dcf7235a6257753aaadcf45ec33c98" Jul 29 01:15:09.961850 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.961827 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8"] Jul 29 01:15:09.965710 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:09.965690 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-a1744c-openshift-default-77bd49876-795q8"] Jul 29 01:15:10.789016 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:10.788970 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" path="/var/lib/kubelet/pods/4140611d-c2ac-4b7e-84e0-3acd80bd0b88/volumes" Jul 29 01:15:22.010777 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:22.010741 2558 generic.go:356] "Generic (PLEG): container finished" podID="72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" containerID="f2294ebd77cb72af2297eeea5ed7a820306af6ed4bf23185249d59d9c2fe96cb" exitCode=0 Jul 29 01:15:22.011212 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:22.010799 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" event={"ID":"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b","Type":"ContainerDied","Data":"f2294ebd77cb72af2297eeea5ed7a820306af6ed4bf23185249d59d9c2fe96cb"} Jul 29 01:15:22.011212 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:22.011148 2558 scope.go:117] "RemoveContainer" containerID="f2294ebd77cb72af2297eeea5ed7a820306af6ed4bf23185249d59d9c2fe96cb" Jul 29 01:15:22.012174 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:22.012142 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:15:22.012282 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:22.012219 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:15:22.012282 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:22.012252 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:15:22.921815 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:22.921778 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:23.017859 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:23.017824 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" event={"ID":"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b","Type":"ContainerStarted","Data":"268302e7ed6eefb63f0fad7448013cfc8508486f0a8d222fb5caa3615423811d"} Jul 29 01:15:23.972754 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:23.972715 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:23.973759 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:23.973724 2558 patch_prober.go:28] interesting pod/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 29 01:15:23.973900 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:23.973783 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" podUID="72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 29 01:15:24.973064 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:24.973024 2558 patch_prober.go:28] interesting pod/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 29 01:15:24.973675 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:24.973644 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" podUID="72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 29 01:15:25.973302 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:25.973262 2558 patch_prober.go:28] interesting pod/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 29 01:15:25.974301 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:25.973907 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" podUID="72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 29 01:15:26.977036 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:26.977003 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:26.977523 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:26.977421 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:26.977990 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:26.977971 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:15:33.548164 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.548131 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm"] Jul 29 01:15:33.548563 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.548548 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerName="istio-proxy" Jul 29 01:15:33.548856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.548566 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerName="istio-proxy" Jul 29 01:15:33.548856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.548585 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerName="istio-proxy" Jul 29 01:15:33.548856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.548591 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerName="istio-proxy" Jul 29 01:15:33.548856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.548654 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="61906f63-5663-42d0-adc8-abd2dba31083" containerName="istio-proxy" Jul 29 01:15:33.548856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.548661 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerName="istio-proxy" Jul 29 01:15:33.548856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.548669 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4140611d-c2ac-4b7e-84e0-3acd80bd0b88" containerName="istio-proxy" Jul 29 01:15:33.552046 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.552024 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.554144 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.554117 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-d5d9a9-openshift-default-dockercfg-4sf8h\"" Jul 29 01:15:33.563873 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.563813 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm"] Jul 29 01:15:33.637409 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.637361 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-workload-socket\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.637559 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.637460 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-envoy\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.637559 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.637526 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-workload-certs\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.637559 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.637547 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-data\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.637763 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.637583 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-credential-socket\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.637763 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.637609 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istiod-ca-cert\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.637763 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.637642 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-podinfo\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.637763 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.637693 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-token\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.637964 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.637775 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7r2q\" (UniqueName: \"kubernetes.io/projected/a843d4db-20cb-44fd-a148-8a5ee023f9ea-kube-api-access-z7r2q\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.738666 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.738631 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-workload-socket\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.738844 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.738673 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-envoy\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.738844 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.738717 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-workload-certs\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.738844 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.738740 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-data\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.738844 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.738770 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-credential-socket\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.738844 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.738797 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istiod-ca-cert\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.739129 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.738849 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-podinfo\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.739129 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.739081 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-workload-socket\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.739244 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.739173 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-token\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.739297 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.739240 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7r2q\" (UniqueName: \"kubernetes.io/projected/a843d4db-20cb-44fd-a148-8a5ee023f9ea-kube-api-access-z7r2q\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.739297 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.739262 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-workload-certs\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.739477 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.739238 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-data\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.739555 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.739534 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-credential-socket\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.739714 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.739690 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istiod-ca-cert\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.741087 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.741063 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-envoy\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.741342 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.741322 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-podinfo\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.746528 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.746506 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-token\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.746714 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.746693 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7r2q\" (UniqueName: \"kubernetes.io/projected/a843d4db-20cb-44fd-a148-8a5ee023f9ea-kube-api-access-z7r2q\") pod \"e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:33.865922 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:33.865894 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:34.003333 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:34.003295 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm"] Jul 29 01:15:34.005289 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:15:34.005255 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda843d4db_20cb_44fd_a148_8a5ee023f9ea.slice/crio-8a3341c712694f53d8c35993956dc4a764cebaca0c2fca5c338493692c87d323 WatchSource:0}: Error finding container 8a3341c712694f53d8c35993956dc4a764cebaca0c2fca5c338493692c87d323: Status 404 returned error can't find the container with id 8a3341c712694f53d8c35993956dc4a764cebaca0c2fca5c338493692c87d323 Jul 29 01:15:34.007514 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:34.007470 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:15:34.007606 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:34.007572 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:15:34.007653 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:34.007619 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:15:34.075377 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:34.075350 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" event={"ID":"a843d4db-20cb-44fd-a148-8a5ee023f9ea","Type":"ContainerStarted","Data":"28680003385ba527fc0cc10b7c183d5ca8144760c9518ece5a4137f3768fda9e"} Jul 29 01:15:34.075490 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:34.075402 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" event={"ID":"a843d4db-20cb-44fd-a148-8a5ee023f9ea","Type":"ContainerStarted","Data":"8a3341c712694f53d8c35993956dc4a764cebaca0c2fca5c338493692c87d323"} Jul 29 01:15:34.093515 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:34.093461 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" podStartSLOduration=1.093443867 podStartE2EDuration="1.093443867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:15:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:15:34.092139334 +0000 UTC m=+2385.751365228" watchObservedRunningTime="2026-07-29 01:15:34.093443867 +0000 UTC m=+2385.752669759" Jul 29 01:15:34.867034 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:34.866996 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:35.872115 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:35.872084 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:36.085215 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:36.085174 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:36.086130 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:36.086111 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:48.917655 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:48.917620 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:15:48.922296 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:48.922271 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 01:15:48.923724 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:48.923693 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:15:48.928065 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:48.928049 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 01:15:52.126474 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:52.126435 2558 patch_prober.go:28] interesting pod/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 29 01:15:52.126880 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:52.126512 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" podUID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 29 01:15:53.166162 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:53.166129 2558 generic.go:356] "Generic (PLEG): container finished" podID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" containerID="28680003385ba527fc0cc10b7c183d5ca8144760c9518ece5a4137f3768fda9e" exitCode=0 Jul 29 01:15:53.166591 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:53.166201 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" event={"ID":"a843d4db-20cb-44fd-a148-8a5ee023f9ea","Type":"ContainerDied","Data":"28680003385ba527fc0cc10b7c183d5ca8144760c9518ece5a4137f3768fda9e"} Jul 29 01:15:53.166680 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:53.166663 2558 scope.go:117] "RemoveContainer" containerID="28680003385ba527fc0cc10b7c183d5ca8144760c9518ece5a4137f3768fda9e" Jul 29 01:15:53.167715 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:53.167686 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:15:53.167776 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:53.167755 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:15:53.167812 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:53.167785 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:15:54.173151 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:54.173117 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" event={"ID":"a843d4db-20cb-44fd-a148-8a5ee023f9ea","Type":"ContainerStarted","Data":"56341414442ebf259067ff5e80ce2e6e08fbdaaddb82361b32de6df0e94671d6"} Jul 29 01:15:54.866396 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:54.866356 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:54.866710 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:54.866673 2558 patch_prober.go:28] interesting pod/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 29 01:15:54.866805 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:54.866754 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" podUID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 29 01:15:55.867360 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:55.866900 2558 patch_prober.go:28] interesting pod/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 29 01:15:55.867360 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:55.866960 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" podUID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 29 01:15:56.867236 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:56.867198 2558 patch_prober.go:28] interesting pod/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 29 01:15:56.867549 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:56.867513 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" podUID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 29 01:15:57.866857 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:57.866818 2558 patch_prober.go:28] interesting pod/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 29 01:15:57.867164 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:57.867132 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" podUID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 29 01:15:58.869653 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:58.869616 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:58.870115 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:58.869866 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:15:58.870496 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:15:58.870480 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:16:25.846383 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:25.846281 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx"] Jul 29 01:16:25.846918 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:25.846587 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" podUID="72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" containerName="istio-proxy" containerID="cri-o://268302e7ed6eefb63f0fad7448013cfc8508486f0a8d222fb5caa3615423811d" gracePeriod=30 Jul 29 01:16:26.979961 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:26.979919 2558 patch_prober.go:28] interesting pod/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 01:16:26.980444 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:26.979987 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" podUID="72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 01:16:31.103803 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.103777 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:16:31.172643 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.172575 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-credential-socket\") pod \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " Jul 29 01:16:31.172643 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.172638 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrbr8\" (UniqueName: \"kubernetes.io/projected/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-kube-api-access-hrbr8\") pod \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " Jul 29 01:16:31.172815 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.172658 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-token\") pod \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " Jul 29 01:16:31.172815 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.172708 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-envoy\") pod \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " Jul 29 01:16:31.172815 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.172753 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-workload-socket\") pod \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " Jul 29 01:16:31.172815 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.172778 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-podinfo\") pod \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " Jul 29 01:16:31.172815 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.172801 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-data\") pod \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " Jul 29 01:16:31.173084 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.172821 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istiod-ca-cert\") pod \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " Jul 29 01:16:31.173084 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.172852 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-workload-certs\") pod \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\" (UID: \"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b\") " Jul 29 01:16:31.173191 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.172846 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" (UID: "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:16:31.173191 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.173126 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" (UID: "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:16:31.173191 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.173073 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" (UID: "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:16:31.173191 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.173179 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" (UID: "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:16:31.173657 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.173632 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-data" (OuterVolumeSpecName: "istio-data") pod "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" (UID: "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:16:31.174891 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.174865 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-token" (OuterVolumeSpecName: "istio-token") pod "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" (UID: "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:16:31.175119 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.175091 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-kube-api-access-hrbr8" (OuterVolumeSpecName: "kube-api-access-hrbr8") pod "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" (UID: "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b"). InnerVolumeSpecName "kube-api-access-hrbr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:16:31.175350 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.175327 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" (UID: "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:16:31.175415 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.175328 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" (UID: "72e37be6-ed6d-4a8a-ab67-aa4005c0e05b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 01:16:31.273694 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.273667 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-workload-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:31.273694 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.273693 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-podinfo\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:31.273879 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.273706 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-data\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:31.273879 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.273714 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istiod-ca-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:31.273879 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.273722 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-workload-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:31.273879 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.273729 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-credential-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:31.273879 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.273738 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrbr8\" (UniqueName: \"kubernetes.io/projected/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-kube-api-access-hrbr8\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:31.273879 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.273747 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:31.273879 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.273754 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b-istio-envoy\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:31.354357 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.354304 2558 generic.go:356] "Generic (PLEG): container finished" podID="72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" containerID="268302e7ed6eefb63f0fad7448013cfc8508486f0a8d222fb5caa3615423811d" exitCode=0 Jul 29 01:16:31.354475 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.354370 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" Jul 29 01:16:31.354475 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.354417 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" event={"ID":"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b","Type":"ContainerDied","Data":"268302e7ed6eefb63f0fad7448013cfc8508486f0a8d222fb5caa3615423811d"} Jul 29 01:16:31.354562 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.354502 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx" event={"ID":"72e37be6-ed6d-4a8a-ab67-aa4005c0e05b","Type":"ContainerDied","Data":"69cf755c267b472db8befa127b1fd027de5d68df029304a0f40334765ff71281"} Jul 29 01:16:31.354562 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.354530 2558 scope.go:117] "RemoveContainer" containerID="268302e7ed6eefb63f0fad7448013cfc8508486f0a8d222fb5caa3615423811d" Jul 29 01:16:31.364216 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.364195 2558 scope.go:117] "RemoveContainer" containerID="f2294ebd77cb72af2297eeea5ed7a820306af6ed4bf23185249d59d9c2fe96cb" Jul 29 01:16:31.372050 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.372030 2558 scope.go:117] "RemoveContainer" containerID="268302e7ed6eefb63f0fad7448013cfc8508486f0a8d222fb5caa3615423811d" Jul 29 01:16:31.372282 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:16:31.372262 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"268302e7ed6eefb63f0fad7448013cfc8508486f0a8d222fb5caa3615423811d\": container with ID starting with 268302e7ed6eefb63f0fad7448013cfc8508486f0a8d222fb5caa3615423811d not found: ID does not exist" containerID="268302e7ed6eefb63f0fad7448013cfc8508486f0a8d222fb5caa3615423811d" Jul 29 01:16:31.372357 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.372290 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"268302e7ed6eefb63f0fad7448013cfc8508486f0a8d222fb5caa3615423811d"} err="failed to get container status \"268302e7ed6eefb63f0fad7448013cfc8508486f0a8d222fb5caa3615423811d\": rpc error: code = NotFound desc = could not find container \"268302e7ed6eefb63f0fad7448013cfc8508486f0a8d222fb5caa3615423811d\": container with ID starting with 268302e7ed6eefb63f0fad7448013cfc8508486f0a8d222fb5caa3615423811d not found: ID does not exist" Jul 29 01:16:31.372357 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.372319 2558 scope.go:117] "RemoveContainer" containerID="f2294ebd77cb72af2297eeea5ed7a820306af6ed4bf23185249d59d9c2fe96cb" Jul 29 01:16:31.372572 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:16:31.372550 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2294ebd77cb72af2297eeea5ed7a820306af6ed4bf23185249d59d9c2fe96cb\": container with ID starting with f2294ebd77cb72af2297eeea5ed7a820306af6ed4bf23185249d59d9c2fe96cb not found: ID does not exist" containerID="f2294ebd77cb72af2297eeea5ed7a820306af6ed4bf23185249d59d9c2fe96cb" Jul 29 01:16:31.372673 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.372579 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2294ebd77cb72af2297eeea5ed7a820306af6ed4bf23185249d59d9c2fe96cb"} err="failed to get container status \"f2294ebd77cb72af2297eeea5ed7a820306af6ed4bf23185249d59d9c2fe96cb\": rpc error: code = NotFound desc = could not find container \"f2294ebd77cb72af2297eeea5ed7a820306af6ed4bf23185249d59d9c2fe96cb\": container with ID starting with f2294ebd77cb72af2297eeea5ed7a820306af6ed4bf23185249d59d9c2fe96cb not found: ID does not exist" Jul 29 01:16:31.374937 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.374918 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx"] Jul 29 01:16:31.378420 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:31.378399 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-4650f4-openshift-default-68bc6cbb74-t66xx"] Jul 29 01:16:32.789777 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:32.789744 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" path="/var/lib/kubelet/pods/72e37be6-ed6d-4a8a-ab67-aa4005c0e05b/volumes" Jul 29 01:16:48.272772 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:48.272736 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm"] Jul 29 01:16:48.273240 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:48.273001 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" podUID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" containerName="istio-proxy" containerID="cri-o://56341414442ebf259067ff5e80ce2e6e08fbdaaddb82361b32de6df0e94671d6" gracePeriod=30 Jul 29 01:16:49.137799 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.137766 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd"] Jul 29 01:16:49.138174 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.138160 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" containerName="istio-proxy" Jul 29 01:16:49.138239 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.138176 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" containerName="istio-proxy" Jul 29 01:16:49.138239 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.138189 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" containerName="istio-proxy" Jul 29 01:16:49.138239 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.138196 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" containerName="istio-proxy" Jul 29 01:16:49.138444 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.138254 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" containerName="istio-proxy" Jul 29 01:16:49.138444 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.138263 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="72e37be6-ed6d-4a8a-ab67-aa4005c0e05b" containerName="istio-proxy" Jul 29 01:16:49.143134 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.143112 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.145278 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.145260 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-ebcf3a-openshift-default-dockercfg-m6fzw\"" Jul 29 01:16:49.154079 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.154054 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd"] Jul 29 01:16:49.224320 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.224272 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-envoy\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.224481 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.224334 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn764\" (UniqueName: \"kubernetes.io/projected/73708d90-9fcf-4be7-a22b-466406b2df4c-kube-api-access-bn764\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.224481 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.224366 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-workload-certs\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.224481 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.224394 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-credential-socket\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.224624 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.224491 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73708d90-9fcf-4be7-a22b-466406b2df4c-istiod-ca-cert\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.224624 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.224531 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-podinfo\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.224624 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.224610 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-data\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.224768 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.224669 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-token\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.224822 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.224757 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-workload-socket\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.325369 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.325332 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-token\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.325856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.325402 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-workload-socket\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.325856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.325429 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-envoy\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.325856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.325445 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn764\" (UniqueName: \"kubernetes.io/projected/73708d90-9fcf-4be7-a22b-466406b2df4c-kube-api-access-bn764\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.325856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.325471 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-workload-certs\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.325856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.325498 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-credential-socket\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.325856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.325542 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73708d90-9fcf-4be7-a22b-466406b2df4c-istiod-ca-cert\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.325856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.325573 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-podinfo\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.325856 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.325615 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-data\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.326174 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.325876 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-workload-socket\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.326174 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.325968 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-workload-certs\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.326174 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.326029 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-data\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.326174 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.326103 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-credential-socket\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.326397 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.326374 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73708d90-9fcf-4be7-a22b-466406b2df4c-istiod-ca-cert\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.328193 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.328169 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-envoy\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.328380 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.328362 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-podinfo\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.340049 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.340028 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-token\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.343106 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.343081 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn764\" (UniqueName: \"kubernetes.io/projected/73708d90-9fcf-4be7-a22b-466406b2df4c-kube-api-access-bn764\") pod \"e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.454541 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.454516 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:49.589953 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.589929 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd"] Jul 29 01:16:49.591723 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:16:49.591695 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73708d90_9fcf_4be7_a22b_466406b2df4c.slice/crio-f218627653f128f1766ce1c78638f55407c5946230a2cd7368515ede6edf2eca WatchSource:0}: Error finding container f218627653f128f1766ce1c78638f55407c5946230a2cd7368515ede6edf2eca: Status 404 returned error can't find the container with id f218627653f128f1766ce1c78638f55407c5946230a2cd7368515ede6edf2eca Jul 29 01:16:49.593780 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.593744 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:16:49.593864 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.593814 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:16:49.593864 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:49.593847 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:16:50.448390 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:50.448352 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" event={"ID":"73708d90-9fcf-4be7-a22b-466406b2df4c","Type":"ContainerStarted","Data":"c46b0b746466d3f638d6b192b672ec1c69bdc264896cd5a1226c61b1ab1fa3b5"} Jul 29 01:16:50.448771 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:50.448413 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" event={"ID":"73708d90-9fcf-4be7-a22b-466406b2df4c","Type":"ContainerStarted","Data":"f218627653f128f1766ce1c78638f55407c5946230a2cd7368515ede6edf2eca"} Jul 29 01:16:50.454936 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:50.454907 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:50.459798 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:50.459771 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:50.466155 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:50.466108 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" podStartSLOduration=1.466095797 podStartE2EDuration="1.466095797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:16:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:16:50.465134848 +0000 UTC m=+2462.124360778" watchObservedRunningTime="2026-07-29 01:16:50.466095797 +0000 UTC m=+2462.125321689" Jul 29 01:16:51.453216 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:51.453185 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:51.454038 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:51.454020 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:16:53.465195 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.465166 2558 generic.go:356] "Generic (PLEG): container finished" podID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" containerID="56341414442ebf259067ff5e80ce2e6e08fbdaaddb82361b32de6df0e94671d6" exitCode=0 Jul 29 01:16:53.465649 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.465237 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" event={"ID":"a843d4db-20cb-44fd-a148-8a5ee023f9ea","Type":"ContainerDied","Data":"56341414442ebf259067ff5e80ce2e6e08fbdaaddb82361b32de6df0e94671d6"} Jul 29 01:16:53.465649 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.465297 2558 scope.go:117] "RemoveContainer" containerID="28680003385ba527fc0cc10b7c183d5ca8144760c9518ece5a4137f3768fda9e" Jul 29 01:16:53.530107 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.530084 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:16:53.669642 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.669616 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7r2q\" (UniqueName: \"kubernetes.io/projected/a843d4db-20cb-44fd-a148-8a5ee023f9ea-kube-api-access-z7r2q\") pod \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " Jul 29 01:16:53.669820 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.669688 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-workload-certs\") pod \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " Jul 29 01:16:53.669820 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.669708 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istiod-ca-cert\") pod \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " Jul 29 01:16:53.669820 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.669743 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-podinfo\") pod \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " Jul 29 01:16:53.669820 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.669765 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-token\") pod \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " Jul 29 01:16:53.669820 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.669790 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-workload-socket\") pod \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " Jul 29 01:16:53.670080 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.669828 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-envoy\") pod \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " Jul 29 01:16:53.670080 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.669861 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-credential-socket\") pod \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " Jul 29 01:16:53.670080 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.669894 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-data\") pod \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\" (UID: \"a843d4db-20cb-44fd-a148-8a5ee023f9ea\") " Jul 29 01:16:53.670080 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.670040 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a843d4db-20cb-44fd-a148-8a5ee023f9ea" (UID: "a843d4db-20cb-44fd-a148-8a5ee023f9ea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:16:53.670288 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.670088 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a843d4db-20cb-44fd-a148-8a5ee023f9ea" (UID: "a843d4db-20cb-44fd-a148-8a5ee023f9ea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:16:53.670288 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.670212 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-workload-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:53.670288 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.670232 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istiod-ca-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:53.670485 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.670289 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a843d4db-20cb-44fd-a148-8a5ee023f9ea" (UID: "a843d4db-20cb-44fd-a148-8a5ee023f9ea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:16:53.670485 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.670340 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a843d4db-20cb-44fd-a148-8a5ee023f9ea" (UID: "a843d4db-20cb-44fd-a148-8a5ee023f9ea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:16:53.671033 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.671005 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-data" (OuterVolumeSpecName: "istio-data") pod "a843d4db-20cb-44fd-a148-8a5ee023f9ea" (UID: "a843d4db-20cb-44fd-a148-8a5ee023f9ea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:16:53.672029 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.672006 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-token" (OuterVolumeSpecName: "istio-token") pod "a843d4db-20cb-44fd-a148-8a5ee023f9ea" (UID: "a843d4db-20cb-44fd-a148-8a5ee023f9ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:16:53.672146 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.672130 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a843d4db-20cb-44fd-a148-8a5ee023f9ea" (UID: "a843d4db-20cb-44fd-a148-8a5ee023f9ea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:16:53.672267 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.672250 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a843d4db-20cb-44fd-a148-8a5ee023f9ea-kube-api-access-z7r2q" (OuterVolumeSpecName: "kube-api-access-z7r2q") pod "a843d4db-20cb-44fd-a148-8a5ee023f9ea" (UID: "a843d4db-20cb-44fd-a148-8a5ee023f9ea"). InnerVolumeSpecName "kube-api-access-z7r2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:16:53.672333 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.672304 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a843d4db-20cb-44fd-a148-8a5ee023f9ea" (UID: "a843d4db-20cb-44fd-a148-8a5ee023f9ea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 01:16:53.770661 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.770633 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-podinfo\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:53.770661 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.770657 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:53.770661 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.770666 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-workload-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:53.770863 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.770675 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-envoy\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:53.770863 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.770685 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-credential-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:53.770863 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.770693 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a843d4db-20cb-44fd-a148-8a5ee023f9ea-istio-data\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:53.770863 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:53.770701 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7r2q\" (UniqueName: \"kubernetes.io/projected/a843d4db-20cb-44fd-a148-8a5ee023f9ea-kube-api-access-z7r2q\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:16:54.471576 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:54.471542 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" Jul 29 01:16:54.472023 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:54.471541 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm" event={"ID":"a843d4db-20cb-44fd-a148-8a5ee023f9ea","Type":"ContainerDied","Data":"8a3341c712694f53d8c35993956dc4a764cebaca0c2fca5c338493692c87d323"} Jul 29 01:16:54.472023 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:54.471737 2558 scope.go:117] "RemoveContainer" containerID="56341414442ebf259067ff5e80ce2e6e08fbdaaddb82361b32de6df0e94671d6" Jul 29 01:16:54.490845 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:54.490823 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm"] Jul 29 01:16:54.494866 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:54.494846 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-d5d9a9-openshift-default-7fdd4bcd7d-zw5rm"] Jul 29 01:16:54.789199 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:16:54.789123 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" path="/var/lib/kubelet/pods/a843d4db-20cb-44fd-a148-8a5ee023f9ea/volumes" Jul 29 01:17:06.600357 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:06.599891 2558 patch_prober.go:28] interesting pod/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": read tcp 10.132.0.2:52614->10.132.0.53:15021: read: connection reset by peer" start-of-body= Jul 29 01:17:06.600357 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:06.599956 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" podUID="73708d90-9fcf-4be7-a22b-466406b2df4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": read tcp 10.132.0.2:52614->10.132.0.53:15021: read: connection reset by peer" Jul 29 01:17:07.537953 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:07.537921 2558 generic.go:356] "Generic (PLEG): container finished" podID="73708d90-9fcf-4be7-a22b-466406b2df4c" containerID="c46b0b746466d3f638d6b192b672ec1c69bdc264896cd5a1226c61b1ab1fa3b5" exitCode=0 Jul 29 01:17:07.538120 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:07.537995 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" event={"ID":"73708d90-9fcf-4be7-a22b-466406b2df4c","Type":"ContainerDied","Data":"c46b0b746466d3f638d6b192b672ec1c69bdc264896cd5a1226c61b1ab1fa3b5"} Jul 29 01:17:07.538417 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:07.538404 2558 scope.go:117] "RemoveContainer" containerID="c46b0b746466d3f638d6b192b672ec1c69bdc264896cd5a1226c61b1ab1fa3b5" Jul 29 01:17:07.539389 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:07.539362 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:17:07.539472 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:07.539440 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:17:07.539511 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:07.539472 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:17:08.544325 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:08.544276 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" event={"ID":"73708d90-9fcf-4be7-a22b-466406b2df4c","Type":"ContainerStarted","Data":"5d1c538fa1f8436b6a737c2c58c1a6b024795c4cb8c5edb43ad6c90f6be98ac1"} Jul 29 01:17:09.454751 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:09.454719 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:17:09.455032 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:09.455005 2558 patch_prober.go:28] interesting pod/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 29 01:17:09.455136 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:09.455059 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" podUID="73708d90-9fcf-4be7-a22b-466406b2df4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 29 01:17:10.455395 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:10.455350 2558 patch_prober.go:28] interesting pod/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 29 01:17:10.455872 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:10.455430 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" podUID="73708d90-9fcf-4be7-a22b-466406b2df4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 29 01:17:11.455815 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:11.455769 2558 patch_prober.go:28] interesting pod/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 29 01:17:11.456300 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:11.455840 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" podUID="73708d90-9fcf-4be7-a22b-466406b2df4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 29 01:17:12.458150 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:12.458119 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:17:12.458561 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:12.458412 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:17:12.459115 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:12.459088 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:17:17.927833 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.927786 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj"] Jul 29 01:17:17.929360 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.929335 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" containerName="istio-proxy" Jul 29 01:17:17.929360 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.929360 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" containerName="istio-proxy" Jul 29 01:17:17.929567 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.929398 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" containerName="istio-proxy" Jul 29 01:17:17.929567 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.929407 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" containerName="istio-proxy" Jul 29 01:17:17.929567 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.929530 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" containerName="istio-proxy" Jul 29 01:17:17.929567 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.929543 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a843d4db-20cb-44fd-a148-8a5ee023f9ea" containerName="istio-proxy" Jul 29 01:17:17.932408 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.932385 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:17.935193 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.935173 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-afd6d5-openshift-default-dockercfg-wbxw8\"" Jul 29 01:17:17.944926 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.944900 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj"] Jul 29 01:17:17.991059 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.991026 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istiod-ca-cert\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:17.991271 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.991077 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-workload-certs\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:17.991271 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.991103 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-data\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:17.991271 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.991150 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-podinfo\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:17.991271 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.991165 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-token\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:17.991271 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.991221 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-workload-socket\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:17.991271 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.991249 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-credential-socket\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:17.991271 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.991266 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb7rv\" (UniqueName: \"kubernetes.io/projected/593c24cf-d06a-4a8c-9637-66ef311d4fd0-kube-api-access-jb7rv\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:17.991641 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:17.991318 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-envoy\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.092031 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.091999 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-workload-socket\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.092173 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.092045 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-credential-socket\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.092216 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.092160 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb7rv\" (UniqueName: \"kubernetes.io/projected/593c24cf-d06a-4a8c-9637-66ef311d4fd0-kube-api-access-jb7rv\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.092346 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.092328 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-envoy\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.092388 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.092372 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istiod-ca-cert\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.092445 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.092430 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-workload-certs\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.092501 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.092444 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-credential-socket\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.092501 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.092471 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-workload-socket\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.092501 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.092486 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-data\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.092653 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.092563 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-podinfo\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.092653 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.092586 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-token\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.092766 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.092714 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-workload-certs\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.092822 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.092792 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-data\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.093155 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.093127 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istiod-ca-cert\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.094575 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.094557 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-envoy\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.095034 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.095014 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-podinfo\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.105803 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.105779 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-token\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.106081 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.106057 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb7rv\" (UniqueName: \"kubernetes.io/projected/593c24cf-d06a-4a8c-9637-66ef311d4fd0-kube-api-access-jb7rv\") pod \"e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.245579 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.245489 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:18.382931 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.382907 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj"] Jul 29 01:17:18.385541 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:17:18.385515 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod593c24cf_d06a_4a8c_9637_66ef311d4fd0.slice/crio-669b0b54ca6478d030fd1143495fd4551831e91e2e0b911114a0e8e9195176e2 WatchSource:0}: Error finding container 669b0b54ca6478d030fd1143495fd4551831e91e2e0b911114a0e8e9195176e2: Status 404 returned error can't find the container with id 669b0b54ca6478d030fd1143495fd4551831e91e2e0b911114a0e8e9195176e2 Jul 29 01:17:18.387715 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.387681 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:17:18.387790 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.387748 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:17:18.387790 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.387780 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:17:18.599699 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.599660 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" event={"ID":"593c24cf-d06a-4a8c-9637-66ef311d4fd0","Type":"ContainerStarted","Data":"6129ecd2bd53772b65d1480e6d2f7eceac0334bf254f96dab3a24763e056dcde"} Jul 29 01:17:18.599898 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.599723 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" event={"ID":"593c24cf-d06a-4a8c-9637-66ef311d4fd0","Type":"ContainerStarted","Data":"669b0b54ca6478d030fd1143495fd4551831e91e2e0b911114a0e8e9195176e2"} Jul 29 01:17:18.617830 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:18.617774 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" podStartSLOduration=1.6177563780000002 podStartE2EDuration="1.617756378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:17:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:17:18.615504466 +0000 UTC m=+2490.274730359" watchObservedRunningTime="2026-07-29 01:17:18.617756378 +0000 UTC m=+2490.276982273" Jul 29 01:17:19.246730 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:19.246697 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:19.251586 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:19.251562 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:19.605049 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:19.605018 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:19.605852 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:19.605835 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:33.681247 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:33.681212 2558 generic.go:356] "Generic (PLEG): container finished" podID="593c24cf-d06a-4a8c-9637-66ef311d4fd0" containerID="6129ecd2bd53772b65d1480e6d2f7eceac0334bf254f96dab3a24763e056dcde" exitCode=0 Jul 29 01:17:33.681705 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:33.681283 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" event={"ID":"593c24cf-d06a-4a8c-9637-66ef311d4fd0","Type":"ContainerDied","Data":"6129ecd2bd53772b65d1480e6d2f7eceac0334bf254f96dab3a24763e056dcde"} Jul 29 01:17:33.681752 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:33.681714 2558 scope.go:117] "RemoveContainer" containerID="6129ecd2bd53772b65d1480e6d2f7eceac0334bf254f96dab3a24763e056dcde" Jul 29 01:17:33.682716 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:33.682686 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:17:33.682824 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:33.682755 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:17:33.682824 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:33.682786 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:17:34.605906 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:34.605854 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:34.688483 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:34.688444 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" event={"ID":"593c24cf-d06a-4a8c-9637-66ef311d4fd0","Type":"ContainerStarted","Data":"57333d75bc878c3ddd7283fed702200878ae1c03f1caa1389a891cb0f805cc00"} Jul 29 01:17:35.245745 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:35.245712 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:35.246783 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:35.246757 2558 patch_prober.go:28] interesting pod/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 29 01:17:35.246892 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:35.246810 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" podUID="593c24cf-d06a-4a8c-9637-66ef311d4fd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 29 01:17:36.246918 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:36.246880 2558 patch_prober.go:28] interesting pod/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 29 01:17:36.247416 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:36.246940 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" podUID="593c24cf-d06a-4a8c-9637-66ef311d4fd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 29 01:17:37.245924 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:37.245888 2558 patch_prober.go:28] interesting pod/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 29 01:17:37.246099 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:37.245945 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" podUID="593c24cf-d06a-4a8c-9637-66ef311d4fd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 29 01:17:38.245895 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:38.245840 2558 patch_prober.go:28] interesting pod/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 29 01:17:38.245895 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:38.245895 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" podUID="593c24cf-d06a-4a8c-9637-66ef311d4fd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 29 01:17:39.249954 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:39.249927 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:39.250354 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:39.250182 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:17:39.250919 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:17:39.250901 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:20:48.946040 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:20:48.945954 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:20:48.950333 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:20:48.950290 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 01:20:48.956601 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:20:48.956580 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:20:48.960888 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:20:48.960871 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 01:25:48.975399 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:25:48.975324 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:25:48.980392 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:25:48.980370 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 01:25:48.988017 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:25:48.987998 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:25:48.992258 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:25:48.992242 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 01:26:13.016087 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:13.016044 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9"] Jul 29 01:26:13.019694 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:13.019669 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" Jul 29 01:26:13.021882 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:13.021858 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-e81b9d-dockercfg-cgsf6\"" Jul 29 01:26:13.021970 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:13.021911 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-e81b9d\"" Jul 29 01:26:13.029638 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:13.029614 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9"] Jul 29 01:26:13.030859 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:13.030833 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f127472-abcc-4857-8fde-8f1b88006a3f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9\" (UID: \"3f127472-abcc-4857-8fde-8f1b88006a3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" Jul 29 01:26:13.030972 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:13.030933 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54jwm\" (UniqueName: \"kubernetes.io/projected/3f127472-abcc-4857-8fde-8f1b88006a3f-kube-api-access-54jwm\") pod \"payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9\" (UID: \"3f127472-abcc-4857-8fde-8f1b88006a3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" Jul 29 01:26:13.131985 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:13.131953 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f127472-abcc-4857-8fde-8f1b88006a3f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9\" (UID: \"3f127472-abcc-4857-8fde-8f1b88006a3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" Jul 29 01:26:13.132186 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:13.132058 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54jwm\" (UniqueName: \"kubernetes.io/projected/3f127472-abcc-4857-8fde-8f1b88006a3f-kube-api-access-54jwm\") pod \"payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9\" (UID: \"3f127472-abcc-4857-8fde-8f1b88006a3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" Jul 29 01:26:13.132706 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:13.132687 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f127472-abcc-4857-8fde-8f1b88006a3f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9\" (UID: \"3f127472-abcc-4857-8fde-8f1b88006a3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" Jul 29 01:26:13.139100 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:13.139074 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54jwm\" (UniqueName: \"kubernetes.io/projected/3f127472-abcc-4857-8fde-8f1b88006a3f-kube-api-access-54jwm\") pod \"payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9\" (UID: \"3f127472-abcc-4857-8fde-8f1b88006a3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" Jul 29 01:26:13.331662 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:13.331627 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" Jul 29 01:26:13.669586 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:13.669556 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9"] Jul 29 01:26:13.672675 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:26:13.672650 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f127472_abcc_4857_8fde_8f1b88006a3f.slice/crio-158aec7d1e7ca2059f7b2cf617e214383b27db028e3776634c7f40319d9d4a26 WatchSource:0}: Error finding container 158aec7d1e7ca2059f7b2cf617e214383b27db028e3776634c7f40319d9d4a26: Status 404 returned error can't find the container with id 158aec7d1e7ca2059f7b2cf617e214383b27db028e3776634c7f40319d9d4a26 Jul 29 01:26:14.172228 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:14.172192 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" event={"ID":"3f127472-abcc-4857-8fde-8f1b88006a3f","Type":"ContainerStarted","Data":"c5095d1c54da01c3d75cdd69f9881e501002e0360b147ba9802acba9e69b74d0"} Jul 29 01:26:14.172691 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:14.172242 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" event={"ID":"3f127472-abcc-4857-8fde-8f1b88006a3f","Type":"ContainerStarted","Data":"158aec7d1e7ca2059f7b2cf617e214383b27db028e3776634c7f40319d9d4a26"} Jul 29 01:26:14.172691 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:14.172285 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" Jul 29 01:26:14.187687 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:14.187632 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" podStartSLOduration=2.187614194 podStartE2EDuration="2.187614194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:26:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:26:14.185842633 +0000 UTC m=+3025.845068527" watchObservedRunningTime="2026-07-29 01:26:14.187614194 +0000 UTC m=+3025.846840089" Jul 29 01:26:25.179261 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:25.179228 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" Jul 29 01:26:38.156888 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.156842 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v"] Jul 29 01:26:38.160622 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.160599 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.162565 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.162539 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-7c9ee4-openshift-default-dockercfg-2lb8f\"" Jul 29 01:26:38.170943 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.170924 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v"] Jul 29 01:26:38.259478 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.259439 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-workload-socket\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.259478 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.259481 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-credential-socket\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.259688 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.259556 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-istio-data\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.259688 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.259631 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5905104d-ac22-4352-a818-481eb366b144-istio-token\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.259688 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.259651 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5905104d-ac22-4352-a818-481eb366b144-istio-podinfo\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.259812 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.259688 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5905104d-ac22-4352-a818-481eb366b144-istiod-ca-cert\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.259812 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.259726 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-workload-certs\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.259812 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.259746 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgglh\" (UniqueName: \"kubernetes.io/projected/5905104d-ac22-4352-a818-481eb366b144-kube-api-access-fgglh\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.259812 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.259786 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-istio-envoy\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.361273 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.361231 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-credential-socket\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.361479 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.361287 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-istio-data\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.361479 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.361322 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5905104d-ac22-4352-a818-481eb366b144-istio-token\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.361479 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.361348 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5905104d-ac22-4352-a818-481eb366b144-istio-podinfo\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.361479 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.361386 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5905104d-ac22-4352-a818-481eb366b144-istiod-ca-cert\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.361479 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.361419 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-workload-certs\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.361479 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.361449 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgglh\" (UniqueName: \"kubernetes.io/projected/5905104d-ac22-4352-a818-481eb366b144-kube-api-access-fgglh\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.361813 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.361512 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-istio-envoy\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.361813 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.361551 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-workload-socket\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.361813 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.361789 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-istio-data\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.361973 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.361829 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-workload-certs\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.361973 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.361953 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-credential-socket\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.362047 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.361992 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-workload-socket\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.362220 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.362201 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5905104d-ac22-4352-a818-481eb366b144-istiod-ca-cert\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.363869 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.363847 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5905104d-ac22-4352-a818-481eb366b144-istio-podinfo\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.363980 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.363898 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-istio-envoy\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.368586 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.368561 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5905104d-ac22-4352-a818-481eb366b144-istio-token\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.368687 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.368629 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgglh\" (UniqueName: \"kubernetes.io/projected/5905104d-ac22-4352-a818-481eb366b144-kube-api-access-fgglh\") pod \"e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.472023 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.471940 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:38.611365 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.611340 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v"] Jul 29 01:26:38.613370 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:26:38.613337 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5905104d_ac22_4352_a818_481eb366b144.slice/crio-a6e7f3700459e395b649356c4c0c7e0e3f9de3a945866f4d761aaa6c8a988a73 WatchSource:0}: Error finding container a6e7f3700459e395b649356c4c0c7e0e3f9de3a945866f4d761aaa6c8a988a73: Status 404 returned error can't find the container with id a6e7f3700459e395b649356c4c0c7e0e3f9de3a945866f4d761aaa6c8a988a73 Jul 29 01:26:38.615683 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.615647 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:26:38.615779 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.615718 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:26:38.615779 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:38.615749 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:26:39.293331 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:39.293281 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" event={"ID":"5905104d-ac22-4352-a818-481eb366b144","Type":"ContainerStarted","Data":"1017024bee197652ea1b4ad2709d0c85c96110fdeedcf29402a61ec1e2610591"} Jul 29 01:26:39.293721 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:39.293369 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" event={"ID":"5905104d-ac22-4352-a818-481eb366b144","Type":"ContainerStarted","Data":"a6e7f3700459e395b649356c4c0c7e0e3f9de3a945866f4d761aaa6c8a988a73"} Jul 29 01:26:39.312191 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:39.312138 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" podStartSLOduration=1.312123113 podStartE2EDuration="1.312123113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:26:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:26:39.310392407 +0000 UTC m=+3050.969618303" watchObservedRunningTime="2026-07-29 01:26:39.312123113 +0000 UTC m=+3050.971349005" Jul 29 01:26:39.472542 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:39.472508 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:39.477358 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:39.477331 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:40.298671 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:40.298639 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:40.299803 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:40.299783 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:26:41.988628 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:41.988549 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf"] Jul 29 01:26:41.992417 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:41.992389 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" Jul 29 01:26:41.994247 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:41.994223 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-7c9ee4-dockercfg-5p7cw\"" Jul 29 01:26:41.994371 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:41.994223 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-7c9ee4\"" Jul 29 01:26:42.002601 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.002572 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf"] Jul 29 01:26:42.053843 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.053807 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz"] Jul 29 01:26:42.057316 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.057290 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" Jul 29 01:26:42.065503 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.065481 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz"] Jul 29 01:26:42.096825 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.096790 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b82bc0-df6e-4971-a109-8b98f2f2d21f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf\" (UID: \"d9b82bc0-df6e-4971-a109-8b98f2f2d21f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" Jul 29 01:26:42.096986 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.096840 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdddl\" (UniqueName: \"kubernetes.io/projected/d9b82bc0-df6e-4971-a109-8b98f2f2d21f-kube-api-access-pdddl\") pod \"payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf\" (UID: \"d9b82bc0-df6e-4971-a109-8b98f2f2d21f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" Jul 29 01:26:42.198124 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.198086 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/803fb982-dd5b-4e9a-b7a4-48fc0bbd1290-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz\" (UID: \"803fb982-dd5b-4e9a-b7a4-48fc0bbd1290\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" Jul 29 01:26:42.198347 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.198158 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b82bc0-df6e-4971-a109-8b98f2f2d21f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf\" (UID: \"d9b82bc0-df6e-4971-a109-8b98f2f2d21f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" Jul 29 01:26:42.198347 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.198230 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdddl\" (UniqueName: \"kubernetes.io/projected/d9b82bc0-df6e-4971-a109-8b98f2f2d21f-kube-api-access-pdddl\") pod \"payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf\" (UID: \"d9b82bc0-df6e-4971-a109-8b98f2f2d21f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" Jul 29 01:26:42.198347 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.198331 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzwgc\" (UniqueName: \"kubernetes.io/projected/803fb982-dd5b-4e9a-b7a4-48fc0bbd1290-kube-api-access-zzwgc\") pod \"payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz\" (UID: \"803fb982-dd5b-4e9a-b7a4-48fc0bbd1290\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" Jul 29 01:26:42.198859 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.198841 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b82bc0-df6e-4971-a109-8b98f2f2d21f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf\" (UID: \"d9b82bc0-df6e-4971-a109-8b98f2f2d21f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" Jul 29 01:26:42.205591 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.205567 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdddl\" (UniqueName: \"kubernetes.io/projected/d9b82bc0-df6e-4971-a109-8b98f2f2d21f-kube-api-access-pdddl\") pod \"payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf\" (UID: \"d9b82bc0-df6e-4971-a109-8b98f2f2d21f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" Jul 29 01:26:42.299425 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.299346 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzwgc\" (UniqueName: \"kubernetes.io/projected/803fb982-dd5b-4e9a-b7a4-48fc0bbd1290-kube-api-access-zzwgc\") pod \"payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz\" (UID: \"803fb982-dd5b-4e9a-b7a4-48fc0bbd1290\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" Jul 29 01:26:42.299425 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.299420 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/803fb982-dd5b-4e9a-b7a4-48fc0bbd1290-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz\" (UID: \"803fb982-dd5b-4e9a-b7a4-48fc0bbd1290\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" Jul 29 01:26:42.300098 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.300076 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/803fb982-dd5b-4e9a-b7a4-48fc0bbd1290-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz\" (UID: \"803fb982-dd5b-4e9a-b7a4-48fc0bbd1290\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" Jul 29 01:26:42.304274 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.304257 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" Jul 29 01:26:42.306802 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.306779 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzwgc\" (UniqueName: \"kubernetes.io/projected/803fb982-dd5b-4e9a-b7a4-48fc0bbd1290-kube-api-access-zzwgc\") pod \"payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz\" (UID: \"803fb982-dd5b-4e9a-b7a4-48fc0bbd1290\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" Jul 29 01:26:42.368295 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.368270 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" Jul 29 01:26:42.453796 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.453769 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf"] Jul 29 01:26:42.456421 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:26:42.456295 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b82bc0_df6e_4971_a109_8b98f2f2d21f.slice/crio-083830025d6cc30cad9b38f269fbac35e5dffc40bb6642f77c366b1a1226ca21 WatchSource:0}: Error finding container 083830025d6cc30cad9b38f269fbac35e5dffc40bb6642f77c366b1a1226ca21: Status 404 returned error can't find the container with id 083830025d6cc30cad9b38f269fbac35e5dffc40bb6642f77c366b1a1226ca21 Jul 29 01:26:42.522958 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:42.522934 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz"] Jul 29 01:26:42.525099 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:26:42.525063 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod803fb982_dd5b_4e9a_b7a4_48fc0bbd1290.slice/crio-5aadd4ed1908270e1d86b43cc69287f91f22c8db0bbb2bb03dfcb2418e8cdc1b WatchSource:0}: Error finding container 5aadd4ed1908270e1d86b43cc69287f91f22c8db0bbb2bb03dfcb2418e8cdc1b: Status 404 returned error can't find the container with id 5aadd4ed1908270e1d86b43cc69287f91f22c8db0bbb2bb03dfcb2418e8cdc1b Jul 29 01:26:43.314795 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:43.314760 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" event={"ID":"d9b82bc0-df6e-4971-a109-8b98f2f2d21f","Type":"ContainerStarted","Data":"efd254cde3641046d7268660d2f8268d474ee4217fb3d3256249ae52ebb351ec"} Jul 29 01:26:43.315283 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:43.314815 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" event={"ID":"d9b82bc0-df6e-4971-a109-8b98f2f2d21f","Type":"ContainerStarted","Data":"083830025d6cc30cad9b38f269fbac35e5dffc40bb6642f77c366b1a1226ca21"} Jul 29 01:26:43.315283 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:43.314924 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" Jul 29 01:26:43.316255 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:43.316234 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" event={"ID":"803fb982-dd5b-4e9a-b7a4-48fc0bbd1290","Type":"ContainerStarted","Data":"e2049f828365b8158601d5b378c03a1af0968dbf6c0855440387eb4aa47ace54"} Jul 29 01:26:43.316351 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:43.316274 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" event={"ID":"803fb982-dd5b-4e9a-b7a4-48fc0bbd1290","Type":"ContainerStarted","Data":"5aadd4ed1908270e1d86b43cc69287f91f22c8db0bbb2bb03dfcb2418e8cdc1b"} Jul 29 01:26:43.316403 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:43.316348 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" Jul 29 01:26:43.328411 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:43.328368 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" podStartSLOduration=2.32835597 podStartE2EDuration="2.32835597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:26:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:26:43.326961029 +0000 UTC m=+3054.986186923" watchObservedRunningTime="2026-07-29 01:26:43.32835597 +0000 UTC m=+3054.987581905" Jul 29 01:26:43.341775 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:43.341740 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" podStartSLOduration=1.341726964 podStartE2EDuration="1.341726964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:26:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:26:43.339418958 +0000 UTC m=+3054.998644850" watchObservedRunningTime="2026-07-29 01:26:43.341726964 +0000 UTC m=+3055.000952856" Jul 29 01:26:54.324501 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:54.324471 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" Jul 29 01:26:54.324918 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:54.324677 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" Jul 29 01:26:59.426574 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:59.426540 2558 generic.go:356] "Generic (PLEG): container finished" podID="5905104d-ac22-4352-a818-481eb366b144" containerID="1017024bee197652ea1b4ad2709d0c85c96110fdeedcf29402a61ec1e2610591" exitCode=0 Jul 29 01:26:59.427043 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:59.426588 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" event={"ID":"5905104d-ac22-4352-a818-481eb366b144","Type":"ContainerDied","Data":"1017024bee197652ea1b4ad2709d0c85c96110fdeedcf29402a61ec1e2610591"} Jul 29 01:26:59.427043 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:59.426931 2558 scope.go:117] "RemoveContainer" containerID="1017024bee197652ea1b4ad2709d0c85c96110fdeedcf29402a61ec1e2610591" Jul 29 01:26:59.428002 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:59.427972 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:26:59.428063 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:59.428040 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:26:59.428139 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:26:59.428071 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 29 01:27:00.433975 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:27:00.433942 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" event={"ID":"5905104d-ac22-4352-a818-481eb366b144","Type":"ContainerStarted","Data":"b833d40f2ce7839dac1b65f6a22385e2159ab0250d2d70a51d0531087793b86d"} Jul 29 01:27:00.472356 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:27:00.472300 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:27:00.472667 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:27:00.472637 2558 patch_prober.go:28] interesting pod/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 29 01:27:00.472753 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:27:00.472700 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" podUID="5905104d-ac22-4352-a818-481eb366b144" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 29 01:27:01.472762 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:27:01.472719 2558 patch_prober.go:28] interesting pod/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 29 01:27:01.473414 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:27:01.473380 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" podUID="5905104d-ac22-4352-a818-481eb366b144" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 29 01:27:02.473711 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:27:02.473241 2558 patch_prober.go:28] interesting pod/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 29 01:27:02.473711 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:27:02.473320 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" podUID="5905104d-ac22-4352-a818-481eb366b144" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 29 01:27:03.473321 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:27:03.473267 2558 patch_prober.go:28] interesting pod/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 29 01:27:03.473606 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:27:03.473577 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" podUID="5905104d-ac22-4352-a818-481eb366b144" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 29 01:27:04.476082 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:27:04.476056 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:27:04.476564 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:27:04.476541 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:27:04.477112 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:27:04.477096 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:27:13.276410 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:27:13.276384 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz_803fb982-dd5b-4e9a-b7a4-48fc0bbd1290/payload-processing/0.log" Jul 29 01:28:20.841931 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:20.841837 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz_803fb982-dd5b-4e9a-b7a4-48fc0bbd1290/payload-processing/0.log" Jul 29 01:28:25.690393 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:25.690354 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n"] Jul 29 01:28:25.694451 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:25.694423 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" Jul 29 01:28:25.698885 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:25.698826 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-06c978\"" Jul 29 01:28:25.713129 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:25.713099 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n"] Jul 29 01:28:25.759262 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:25.759213 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dxjz\" (UniqueName: \"kubernetes.io/projected/080cefac-510b-4b69-841d-94bad36a16dc-kube-api-access-2dxjz\") pod \"payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n\" (UID: \"080cefac-510b-4b69-841d-94bad36a16dc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" Jul 29 01:28:25.759442 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:25.759399 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/080cefac-510b-4b69-841d-94bad36a16dc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n\" (UID: \"080cefac-510b-4b69-841d-94bad36a16dc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" Jul 29 01:28:25.860785 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:25.860754 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dxjz\" (UniqueName: \"kubernetes.io/projected/080cefac-510b-4b69-841d-94bad36a16dc-kube-api-access-2dxjz\") pod \"payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n\" (UID: \"080cefac-510b-4b69-841d-94bad36a16dc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" Jul 29 01:28:25.860957 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:25.860851 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/080cefac-510b-4b69-841d-94bad36a16dc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n\" (UID: \"080cefac-510b-4b69-841d-94bad36a16dc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" Jul 29 01:28:25.861611 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:25.861589 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/080cefac-510b-4b69-841d-94bad36a16dc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n\" (UID: \"080cefac-510b-4b69-841d-94bad36a16dc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" Jul 29 01:28:25.884103 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:25.884081 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dxjz\" (UniqueName: \"kubernetes.io/projected/080cefac-510b-4b69-841d-94bad36a16dc-kube-api-access-2dxjz\") pod \"payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n\" (UID: \"080cefac-510b-4b69-841d-94bad36a16dc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" Jul 29 01:28:26.007120 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:26.007036 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" Jul 29 01:28:26.153433 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:26.153331 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n"] Jul 29 01:28:26.156383 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:28:26.156351 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod080cefac_510b_4b69_841d_94bad36a16dc.slice/crio-f7a7fa00530b241f14a5694c537a6a3223d23ab1e52eb181fecbafac7eb2fa10 WatchSource:0}: Error finding container f7a7fa00530b241f14a5694c537a6a3223d23ab1e52eb181fecbafac7eb2fa10: Status 404 returned error can't find the container with id f7a7fa00530b241f14a5694c537a6a3223d23ab1e52eb181fecbafac7eb2fa10 Jul 29 01:28:26.875210 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:26.873340 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" event={"ID":"080cefac-510b-4b69-841d-94bad36a16dc","Type":"ContainerStarted","Data":"c561d57303dfaa9718543357aa1c2c72f41f451a802baf0027a60a128222818c"} Jul 29 01:28:26.875210 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:26.873466 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" event={"ID":"080cefac-510b-4b69-841d-94bad36a16dc","Type":"ContainerStarted","Data":"f7a7fa00530b241f14a5694c537a6a3223d23ab1e52eb181fecbafac7eb2fa10"} Jul 29 01:28:26.875210 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:26.874459 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" Jul 29 01:28:26.900180 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:26.900125 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" podStartSLOduration=1.9001078599999999 podStartE2EDuration="1.90010786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:28:26.896174799 +0000 UTC m=+3158.555400695" watchObservedRunningTime="2026-07-29 01:28:26.90010786 +0000 UTC m=+3158.559333756" Jul 29 01:28:38.887030 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:38.887000 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" Jul 29 01:28:55.653104 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:55.653063 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n"] Jul 29 01:28:55.653730 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:55.653353 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" podUID="080cefac-510b-4b69-841d-94bad36a16dc" containerName="payload-processing" containerID="cri-o://c561d57303dfaa9718543357aa1c2c72f41f451a802baf0027a60a128222818c" gracePeriod=30 Jul 29 01:28:55.923201 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:55.923177 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" Jul 29 01:28:55.937193 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:55.937165 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/080cefac-510b-4b69-841d-94bad36a16dc-plugins-config-volume\") pod \"080cefac-510b-4b69-841d-94bad36a16dc\" (UID: \"080cefac-510b-4b69-841d-94bad36a16dc\") " Jul 29 01:28:55.937336 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:55.937201 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dxjz\" (UniqueName: \"kubernetes.io/projected/080cefac-510b-4b69-841d-94bad36a16dc-kube-api-access-2dxjz\") pod \"080cefac-510b-4b69-841d-94bad36a16dc\" (UID: \"080cefac-510b-4b69-841d-94bad36a16dc\") " Jul 29 01:28:55.937564 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:55.937539 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/080cefac-510b-4b69-841d-94bad36a16dc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "080cefac-510b-4b69-841d-94bad36a16dc" (UID: "080cefac-510b-4b69-841d-94bad36a16dc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:28:55.939714 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:55.939684 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/080cefac-510b-4b69-841d-94bad36a16dc-kube-api-access-2dxjz" (OuterVolumeSpecName: "kube-api-access-2dxjz") pod "080cefac-510b-4b69-841d-94bad36a16dc" (UID: "080cefac-510b-4b69-841d-94bad36a16dc"). InnerVolumeSpecName "kube-api-access-2dxjz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:28:56.023664 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:56.023625 2558 generic.go:356] "Generic (PLEG): container finished" podID="080cefac-510b-4b69-841d-94bad36a16dc" containerID="c561d57303dfaa9718543357aa1c2c72f41f451a802baf0027a60a128222818c" exitCode=0 Jul 29 01:28:56.023852 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:56.023669 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" Jul 29 01:28:56.023852 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:56.023713 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" event={"ID":"080cefac-510b-4b69-841d-94bad36a16dc","Type":"ContainerDied","Data":"c561d57303dfaa9718543357aa1c2c72f41f451a802baf0027a60a128222818c"} Jul 29 01:28:56.023852 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:56.023766 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n" event={"ID":"080cefac-510b-4b69-841d-94bad36a16dc","Type":"ContainerDied","Data":"f7a7fa00530b241f14a5694c537a6a3223d23ab1e52eb181fecbafac7eb2fa10"} Jul 29 01:28:56.023852 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:56.023790 2558 scope.go:117] "RemoveContainer" containerID="c561d57303dfaa9718543357aa1c2c72f41f451a802baf0027a60a128222818c" Jul 29 01:28:56.033252 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:56.033235 2558 scope.go:117] "RemoveContainer" containerID="c561d57303dfaa9718543357aa1c2c72f41f451a802baf0027a60a128222818c" Jul 29 01:28:56.033493 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:28:56.033474 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c561d57303dfaa9718543357aa1c2c72f41f451a802baf0027a60a128222818c\": container with ID starting with c561d57303dfaa9718543357aa1c2c72f41f451a802baf0027a60a128222818c not found: ID does not exist" containerID="c561d57303dfaa9718543357aa1c2c72f41f451a802baf0027a60a128222818c" Jul 29 01:28:56.033552 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:56.033500 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c561d57303dfaa9718543357aa1c2c72f41f451a802baf0027a60a128222818c"} err="failed to get container status \"c561d57303dfaa9718543357aa1c2c72f41f451a802baf0027a60a128222818c\": rpc error: code = NotFound desc = could not find container \"c561d57303dfaa9718543357aa1c2c72f41f451a802baf0027a60a128222818c\": container with ID starting with c561d57303dfaa9718543357aa1c2c72f41f451a802baf0027a60a128222818c not found: ID does not exist" Jul 29 01:28:56.038402 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:56.038382 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/080cefac-510b-4b69-841d-94bad36a16dc-plugins-config-volume\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:28:56.038500 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:56.038403 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dxjz\" (UniqueName: \"kubernetes.io/projected/080cefac-510b-4b69-841d-94bad36a16dc-kube-api-access-2dxjz\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:28:56.044784 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:56.044758 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n"] Jul 29 01:28:56.048760 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:56.048735 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-06c978-b5bd75c5d-snr9n"] Jul 29 01:28:56.790129 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:28:56.790097 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="080cefac-510b-4b69-841d-94bad36a16dc" path="/var/lib/kubelet/pods/080cefac-510b-4b69-841d-94bad36a16dc/volumes" Jul 29 01:29:21.145941 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:21.145907 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9"] Jul 29 01:29:21.146348 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:21.146157 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" podUID="3f127472-abcc-4857-8fde-8f1b88006a3f" containerName="payload-pre-processing" containerID="cri-o://c5095d1c54da01c3d75cdd69f9881e501002e0360b147ba9802acba9e69b74d0" gracePeriod=30 Jul 29 01:29:21.415181 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:21.415157 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" Jul 29 01:29:21.578119 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:21.578082 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f127472-abcc-4857-8fde-8f1b88006a3f-plugins-config-volume\") pod \"3f127472-abcc-4857-8fde-8f1b88006a3f\" (UID: \"3f127472-abcc-4857-8fde-8f1b88006a3f\") " Jul 29 01:29:21.578282 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:21.578139 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54jwm\" (UniqueName: \"kubernetes.io/projected/3f127472-abcc-4857-8fde-8f1b88006a3f-kube-api-access-54jwm\") pod \"3f127472-abcc-4857-8fde-8f1b88006a3f\" (UID: \"3f127472-abcc-4857-8fde-8f1b88006a3f\") " Jul 29 01:29:21.578534 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:21.578506 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f127472-abcc-4857-8fde-8f1b88006a3f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f127472-abcc-4857-8fde-8f1b88006a3f" (UID: "3f127472-abcc-4857-8fde-8f1b88006a3f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:29:21.580255 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:21.580230 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f127472-abcc-4857-8fde-8f1b88006a3f-kube-api-access-54jwm" (OuterVolumeSpecName: "kube-api-access-54jwm") pod "3f127472-abcc-4857-8fde-8f1b88006a3f" (UID: "3f127472-abcc-4857-8fde-8f1b88006a3f"). InnerVolumeSpecName "kube-api-access-54jwm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:29:21.679637 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:21.679557 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f127472-abcc-4857-8fde-8f1b88006a3f-plugins-config-volume\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:29:21.679637 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:21.679584 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54jwm\" (UniqueName: \"kubernetes.io/projected/3f127472-abcc-4857-8fde-8f1b88006a3f-kube-api-access-54jwm\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:29:22.173350 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:22.173293 2558 generic.go:356] "Generic (PLEG): container finished" podID="3f127472-abcc-4857-8fde-8f1b88006a3f" containerID="c5095d1c54da01c3d75cdd69f9881e501002e0360b147ba9802acba9e69b74d0" exitCode=0 Jul 29 01:29:22.173767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:22.173357 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" Jul 29 01:29:22.173767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:22.173377 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" event={"ID":"3f127472-abcc-4857-8fde-8f1b88006a3f","Type":"ContainerDied","Data":"c5095d1c54da01c3d75cdd69f9881e501002e0360b147ba9802acba9e69b74d0"} Jul 29 01:29:22.173767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:22.173429 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9" event={"ID":"3f127472-abcc-4857-8fde-8f1b88006a3f","Type":"ContainerDied","Data":"158aec7d1e7ca2059f7b2cf617e214383b27db028e3776634c7f40319d9d4a26"} Jul 29 01:29:22.173767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:22.173452 2558 scope.go:117] "RemoveContainer" containerID="c5095d1c54da01c3d75cdd69f9881e501002e0360b147ba9802acba9e69b74d0" Jul 29 01:29:22.183040 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:22.183022 2558 scope.go:117] "RemoveContainer" containerID="c5095d1c54da01c3d75cdd69f9881e501002e0360b147ba9802acba9e69b74d0" Jul 29 01:29:22.183291 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:29:22.183274 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5095d1c54da01c3d75cdd69f9881e501002e0360b147ba9802acba9e69b74d0\": container with ID starting with c5095d1c54da01c3d75cdd69f9881e501002e0360b147ba9802acba9e69b74d0 not found: ID does not exist" containerID="c5095d1c54da01c3d75cdd69f9881e501002e0360b147ba9802acba9e69b74d0" Jul 29 01:29:22.183387 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:22.183299 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5095d1c54da01c3d75cdd69f9881e501002e0360b147ba9802acba9e69b74d0"} err="failed to get container status \"c5095d1c54da01c3d75cdd69f9881e501002e0360b147ba9802acba9e69b74d0\": rpc error: code = NotFound desc = could not find container \"c5095d1c54da01c3d75cdd69f9881e501002e0360b147ba9802acba9e69b74d0\": container with ID starting with c5095d1c54da01c3d75cdd69f9881e501002e0360b147ba9802acba9e69b74d0 not found: ID does not exist" Jul 29 01:29:22.194024 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:22.194000 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9"] Jul 29 01:29:22.197797 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:22.197775 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e81b9d-79566c6d9b-gpdj9"] Jul 29 01:29:22.791009 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:29:22.790978 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f127472-abcc-4857-8fde-8f1b88006a3f" path="/var/lib/kubelet/pods/3f127472-abcc-4857-8fde-8f1b88006a3f/volumes" Jul 29 01:30:21.106656 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.106624 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz"] Jul 29 01:30:21.107254 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.106867 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" podUID="803fb982-dd5b-4e9a-b7a4-48fc0bbd1290" containerName="payload-processing" containerID="cri-o://e2049f828365b8158601d5b378c03a1af0968dbf6c0855440387eb4aa47ace54" gracePeriod=30 Jul 29 01:30:21.116753 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.116298 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf"] Jul 29 01:30:21.116753 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.116573 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" podUID="d9b82bc0-df6e-4971-a109-8b98f2f2d21f" containerName="payload-pre-processing" containerID="cri-o://efd254cde3641046d7268660d2f8268d474ee4217fb3d3256249ae52ebb351ec" gracePeriod=30 Jul 29 01:30:21.411343 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.411301 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" Jul 29 01:30:21.414248 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.414227 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" Jul 29 01:30:21.427284 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.427267 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdddl\" (UniqueName: \"kubernetes.io/projected/d9b82bc0-df6e-4971-a109-8b98f2f2d21f-kube-api-access-pdddl\") pod \"d9b82bc0-df6e-4971-a109-8b98f2f2d21f\" (UID: \"d9b82bc0-df6e-4971-a109-8b98f2f2d21f\") " Jul 29 01:30:21.427386 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.427319 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b82bc0-df6e-4971-a109-8b98f2f2d21f-plugins-config-volume\") pod \"d9b82bc0-df6e-4971-a109-8b98f2f2d21f\" (UID: \"d9b82bc0-df6e-4971-a109-8b98f2f2d21f\") " Jul 29 01:30:21.427386 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.427347 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/803fb982-dd5b-4e9a-b7a4-48fc0bbd1290-plugins-config-volume\") pod \"803fb982-dd5b-4e9a-b7a4-48fc0bbd1290\" (UID: \"803fb982-dd5b-4e9a-b7a4-48fc0bbd1290\") " Jul 29 01:30:21.427472 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.427423 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzwgc\" (UniqueName: \"kubernetes.io/projected/803fb982-dd5b-4e9a-b7a4-48fc0bbd1290-kube-api-access-zzwgc\") pod \"803fb982-dd5b-4e9a-b7a4-48fc0bbd1290\" (UID: \"803fb982-dd5b-4e9a-b7a4-48fc0bbd1290\") " Jul 29 01:30:21.427708 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.427682 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9b82bc0-df6e-4971-a109-8b98f2f2d21f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d9b82bc0-df6e-4971-a109-8b98f2f2d21f" (UID: "d9b82bc0-df6e-4971-a109-8b98f2f2d21f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:30:21.427815 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.427748 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/803fb982-dd5b-4e9a-b7a4-48fc0bbd1290-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "803fb982-dd5b-4e9a-b7a4-48fc0bbd1290" (UID: "803fb982-dd5b-4e9a-b7a4-48fc0bbd1290"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:30:21.429773 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.429734 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/803fb982-dd5b-4e9a-b7a4-48fc0bbd1290-kube-api-access-zzwgc" (OuterVolumeSpecName: "kube-api-access-zzwgc") pod "803fb982-dd5b-4e9a-b7a4-48fc0bbd1290" (UID: "803fb982-dd5b-4e9a-b7a4-48fc0bbd1290"). InnerVolumeSpecName "kube-api-access-zzwgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:30:21.429880 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.429783 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9b82bc0-df6e-4971-a109-8b98f2f2d21f-kube-api-access-pdddl" (OuterVolumeSpecName: "kube-api-access-pdddl") pod "d9b82bc0-df6e-4971-a109-8b98f2f2d21f" (UID: "d9b82bc0-df6e-4971-a109-8b98f2f2d21f"). InnerVolumeSpecName "kube-api-access-pdddl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:30:21.470066 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.470028 2558 generic.go:356] "Generic (PLEG): container finished" podID="d9b82bc0-df6e-4971-a109-8b98f2f2d21f" containerID="efd254cde3641046d7268660d2f8268d474ee4217fb3d3256249ae52ebb351ec" exitCode=0 Jul 29 01:30:21.470218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.470079 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" Jul 29 01:30:21.470218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.470102 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" event={"ID":"d9b82bc0-df6e-4971-a109-8b98f2f2d21f","Type":"ContainerDied","Data":"efd254cde3641046d7268660d2f8268d474ee4217fb3d3256249ae52ebb351ec"} Jul 29 01:30:21.470218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.470153 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf" event={"ID":"d9b82bc0-df6e-4971-a109-8b98f2f2d21f","Type":"ContainerDied","Data":"083830025d6cc30cad9b38f269fbac35e5dffc40bb6642f77c366b1a1226ca21"} Jul 29 01:30:21.470218 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.470177 2558 scope.go:117] "RemoveContainer" containerID="efd254cde3641046d7268660d2f8268d474ee4217fb3d3256249ae52ebb351ec" Jul 29 01:30:21.471814 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.471779 2558 generic.go:356] "Generic (PLEG): container finished" podID="803fb982-dd5b-4e9a-b7a4-48fc0bbd1290" containerID="e2049f828365b8158601d5b378c03a1af0968dbf6c0855440387eb4aa47ace54" exitCode=0 Jul 29 01:30:21.471923 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.471815 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" Jul 29 01:30:21.471923 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.471821 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" event={"ID":"803fb982-dd5b-4e9a-b7a4-48fc0bbd1290","Type":"ContainerDied","Data":"e2049f828365b8158601d5b378c03a1af0968dbf6c0855440387eb4aa47ace54"} Jul 29 01:30:21.471923 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.471863 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz" event={"ID":"803fb982-dd5b-4e9a-b7a4-48fc0bbd1290","Type":"ContainerDied","Data":"5aadd4ed1908270e1d86b43cc69287f91f22c8db0bbb2bb03dfcb2418e8cdc1b"} Jul 29 01:30:21.481700 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.481669 2558 scope.go:117] "RemoveContainer" containerID="efd254cde3641046d7268660d2f8268d474ee4217fb3d3256249ae52ebb351ec" Jul 29 01:30:21.482111 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:30:21.481990 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efd254cde3641046d7268660d2f8268d474ee4217fb3d3256249ae52ebb351ec\": container with ID starting with efd254cde3641046d7268660d2f8268d474ee4217fb3d3256249ae52ebb351ec not found: ID does not exist" containerID="efd254cde3641046d7268660d2f8268d474ee4217fb3d3256249ae52ebb351ec" Jul 29 01:30:21.482111 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.482022 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efd254cde3641046d7268660d2f8268d474ee4217fb3d3256249ae52ebb351ec"} err="failed to get container status \"efd254cde3641046d7268660d2f8268d474ee4217fb3d3256249ae52ebb351ec\": rpc error: code = NotFound desc = could not find container \"efd254cde3641046d7268660d2f8268d474ee4217fb3d3256249ae52ebb351ec\": container with ID starting with efd254cde3641046d7268660d2f8268d474ee4217fb3d3256249ae52ebb351ec not found: ID does not exist" Jul 29 01:30:21.482111 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.482046 2558 scope.go:117] "RemoveContainer" containerID="e2049f828365b8158601d5b378c03a1af0968dbf6c0855440387eb4aa47ace54" Jul 29 01:30:21.491689 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.491670 2558 scope.go:117] "RemoveContainer" containerID="e2049f828365b8158601d5b378c03a1af0968dbf6c0855440387eb4aa47ace54" Jul 29 01:30:21.491916 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:30:21.491895 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2049f828365b8158601d5b378c03a1af0968dbf6c0855440387eb4aa47ace54\": container with ID starting with e2049f828365b8158601d5b378c03a1af0968dbf6c0855440387eb4aa47ace54 not found: ID does not exist" containerID="e2049f828365b8158601d5b378c03a1af0968dbf6c0855440387eb4aa47ace54" Jul 29 01:30:21.491954 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.491926 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2049f828365b8158601d5b378c03a1af0968dbf6c0855440387eb4aa47ace54"} err="failed to get container status \"e2049f828365b8158601d5b378c03a1af0968dbf6c0855440387eb4aa47ace54\": rpc error: code = NotFound desc = could not find container \"e2049f828365b8158601d5b378c03a1af0968dbf6c0855440387eb4aa47ace54\": container with ID starting with e2049f828365b8158601d5b378c03a1af0968dbf6c0855440387eb4aa47ace54 not found: ID does not exist" Jul 29 01:30:21.503117 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.503095 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf"] Jul 29 01:30:21.521911 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.521890 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7c9ee4-7579c6d68-dfvkf"] Jul 29 01:30:21.528782 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.528765 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdddl\" (UniqueName: \"kubernetes.io/projected/d9b82bc0-df6e-4971-a109-8b98f2f2d21f-kube-api-access-pdddl\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:21.528862 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.528783 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b82bc0-df6e-4971-a109-8b98f2f2d21f-plugins-config-volume\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:21.528862 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.528794 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/803fb982-dd5b-4e9a-b7a4-48fc0bbd1290-plugins-config-volume\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:21.528862 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.528803 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzwgc\" (UniqueName: \"kubernetes.io/projected/803fb982-dd5b-4e9a-b7a4-48fc0bbd1290-kube-api-access-zzwgc\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:21.539598 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.539573 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz"] Jul 29 01:30:21.555951 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:21.555930 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7c9ee4-7f7d47b7bb-d29tz"] Jul 29 01:30:22.789666 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:22.789633 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="803fb982-dd5b-4e9a-b7a4-48fc0bbd1290" path="/var/lib/kubelet/pods/803fb982-dd5b-4e9a-b7a4-48fc0bbd1290/volumes" Jul 29 01:30:22.790042 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:22.789972 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9b82bc0-df6e-4971-a109-8b98f2f2d21f" path="/var/lib/kubelet/pods/d9b82bc0-df6e-4971-a109-8b98f2f2d21f/volumes" Jul 29 01:30:27.677406 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:27.677371 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v"] Jul 29 01:30:27.677860 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:27.677632 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" podUID="5905104d-ac22-4352-a818-481eb366b144" containerName="istio-proxy" containerID="cri-o://b833d40f2ce7839dac1b65f6a22385e2159ab0250d2d70a51d0531087793b86d" gracePeriod=30 Jul 29 01:30:32.935302 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:32.935272 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:30:33.032210 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032109 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-credential-socket\") pod \"5905104d-ac22-4352-a818-481eb366b144\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " Jul 29 01:30:33.032210 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032151 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgglh\" (UniqueName: \"kubernetes.io/projected/5905104d-ac22-4352-a818-481eb366b144-kube-api-access-fgglh\") pod \"5905104d-ac22-4352-a818-481eb366b144\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " Jul 29 01:30:33.032210 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032184 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5905104d-ac22-4352-a818-481eb366b144-istiod-ca-cert\") pod \"5905104d-ac22-4352-a818-481eb366b144\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " Jul 29 01:30:33.032532 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032247 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5905104d-ac22-4352-a818-481eb366b144-istio-token\") pod \"5905104d-ac22-4352-a818-481eb366b144\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " Jul 29 01:30:33.032532 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032291 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-workload-socket\") pod \"5905104d-ac22-4352-a818-481eb366b144\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " Jul 29 01:30:33.032532 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032358 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-istio-data\") pod \"5905104d-ac22-4352-a818-481eb366b144\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " Jul 29 01:30:33.032532 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032416 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-istio-envoy\") pod \"5905104d-ac22-4352-a818-481eb366b144\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " Jul 29 01:30:33.032532 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032431 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5905104d-ac22-4352-a818-481eb366b144" (UID: "5905104d-ac22-4352-a818-481eb366b144"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:30:33.032532 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032473 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5905104d-ac22-4352-a818-481eb366b144-istio-podinfo\") pod \"5905104d-ac22-4352-a818-481eb366b144\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " Jul 29 01:30:33.032532 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032505 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-workload-certs\") pod \"5905104d-ac22-4352-a818-481eb366b144\" (UID: \"5905104d-ac22-4352-a818-481eb366b144\") " Jul 29 01:30:33.032888 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032602 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5905104d-ac22-4352-a818-481eb366b144" (UID: "5905104d-ac22-4352-a818-481eb366b144"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:30:33.032888 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032710 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5905104d-ac22-4352-a818-481eb366b144-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5905104d-ac22-4352-a818-481eb366b144" (UID: "5905104d-ac22-4352-a818-481eb366b144"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:30:33.032888 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032863 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-credential-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:33.033037 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032889 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5905104d-ac22-4352-a818-481eb366b144-istiod-ca-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:33.033037 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032904 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-workload-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:33.033037 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.032860 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5905104d-ac22-4352-a818-481eb366b144" (UID: "5905104d-ac22-4352-a818-481eb366b144"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:30:33.033540 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.033509 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-istio-data" (OuterVolumeSpecName: "istio-data") pod "5905104d-ac22-4352-a818-481eb366b144" (UID: "5905104d-ac22-4352-a818-481eb366b144"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:30:33.034495 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.034475 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5905104d-ac22-4352-a818-481eb366b144-kube-api-access-fgglh" (OuterVolumeSpecName: "kube-api-access-fgglh") pod "5905104d-ac22-4352-a818-481eb366b144" (UID: "5905104d-ac22-4352-a818-481eb366b144"). InnerVolumeSpecName "kube-api-access-fgglh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:30:33.034663 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.034648 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5905104d-ac22-4352-a818-481eb366b144-istio-token" (OuterVolumeSpecName: "istio-token") pod "5905104d-ac22-4352-a818-481eb366b144" (UID: "5905104d-ac22-4352-a818-481eb366b144"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:30:33.034782 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.034756 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5905104d-ac22-4352-a818-481eb366b144" (UID: "5905104d-ac22-4352-a818-481eb366b144"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:30:33.034852 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.034840 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5905104d-ac22-4352-a818-481eb366b144-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5905104d-ac22-4352-a818-481eb366b144" (UID: "5905104d-ac22-4352-a818-481eb366b144"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 01:30:33.134391 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.134349 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5905104d-ac22-4352-a818-481eb366b144-istio-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:33.134391 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.134384 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-istio-data\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:33.134391 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.134395 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-istio-envoy\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:33.134640 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.134405 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5905104d-ac22-4352-a818-481eb366b144-istio-podinfo\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:33.134640 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.134417 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5905104d-ac22-4352-a818-481eb366b144-workload-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:33.134640 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.134428 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgglh\" (UniqueName: \"kubernetes.io/projected/5905104d-ac22-4352-a818-481eb366b144-kube-api-access-fgglh\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:33.534432 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.534401 2558 generic.go:356] "Generic (PLEG): container finished" podID="5905104d-ac22-4352-a818-481eb366b144" containerID="b833d40f2ce7839dac1b65f6a22385e2159ab0250d2d70a51d0531087793b86d" exitCode=0 Jul 29 01:30:33.534641 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.534446 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" Jul 29 01:30:33.534641 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.534446 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" event={"ID":"5905104d-ac22-4352-a818-481eb366b144","Type":"ContainerDied","Data":"b833d40f2ce7839dac1b65f6a22385e2159ab0250d2d70a51d0531087793b86d"} Jul 29 01:30:33.534641 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.534590 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v" event={"ID":"5905104d-ac22-4352-a818-481eb366b144","Type":"ContainerDied","Data":"a6e7f3700459e395b649356c4c0c7e0e3f9de3a945866f4d761aaa6c8a988a73"} Jul 29 01:30:33.534641 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.534629 2558 scope.go:117] "RemoveContainer" containerID="b833d40f2ce7839dac1b65f6a22385e2159ab0250d2d70a51d0531087793b86d" Jul 29 01:30:33.545691 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.545675 2558 scope.go:117] "RemoveContainer" containerID="1017024bee197652ea1b4ad2709d0c85c96110fdeedcf29402a61ec1e2610591" Jul 29 01:30:33.553576 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.553560 2558 scope.go:117] "RemoveContainer" containerID="b833d40f2ce7839dac1b65f6a22385e2159ab0250d2d70a51d0531087793b86d" Jul 29 01:30:33.553864 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:30:33.553840 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b833d40f2ce7839dac1b65f6a22385e2159ab0250d2d70a51d0531087793b86d\": container with ID starting with b833d40f2ce7839dac1b65f6a22385e2159ab0250d2d70a51d0531087793b86d not found: ID does not exist" containerID="b833d40f2ce7839dac1b65f6a22385e2159ab0250d2d70a51d0531087793b86d" Jul 29 01:30:33.553959 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.553874 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b833d40f2ce7839dac1b65f6a22385e2159ab0250d2d70a51d0531087793b86d"} err="failed to get container status \"b833d40f2ce7839dac1b65f6a22385e2159ab0250d2d70a51d0531087793b86d\": rpc error: code = NotFound desc = could not find container \"b833d40f2ce7839dac1b65f6a22385e2159ab0250d2d70a51d0531087793b86d\": container with ID starting with b833d40f2ce7839dac1b65f6a22385e2159ab0250d2d70a51d0531087793b86d not found: ID does not exist" Jul 29 01:30:33.553959 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.553894 2558 scope.go:117] "RemoveContainer" containerID="1017024bee197652ea1b4ad2709d0c85c96110fdeedcf29402a61ec1e2610591" Jul 29 01:30:33.554209 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:30:33.554176 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1017024bee197652ea1b4ad2709d0c85c96110fdeedcf29402a61ec1e2610591\": container with ID starting with 1017024bee197652ea1b4ad2709d0c85c96110fdeedcf29402a61ec1e2610591 not found: ID does not exist" containerID="1017024bee197652ea1b4ad2709d0c85c96110fdeedcf29402a61ec1e2610591" Jul 29 01:30:33.554460 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.554431 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1017024bee197652ea1b4ad2709d0c85c96110fdeedcf29402a61ec1e2610591"} err="failed to get container status \"1017024bee197652ea1b4ad2709d0c85c96110fdeedcf29402a61ec1e2610591\": rpc error: code = NotFound desc = could not find container \"1017024bee197652ea1b4ad2709d0c85c96110fdeedcf29402a61ec1e2610591\": container with ID starting with 1017024bee197652ea1b4ad2709d0c85c96110fdeedcf29402a61ec1e2610591 not found: ID does not exist" Jul 29 01:30:33.556375 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.556351 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v"] Jul 29 01:30:33.559872 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:33.559852 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-7c9ee4-openshift-default-5b5c46c56b-qnj4v"] Jul 29 01:30:34.790174 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:34.790140 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5905104d-ac22-4352-a818-481eb366b144" path="/var/lib/kubelet/pods/5905104d-ac22-4352-a818-481eb366b144/volumes" Jul 29 01:30:45.088740 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:45.088702 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd"] Jul 29 01:30:45.089136 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:45.089008 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" podUID="73708d90-9fcf-4be7-a22b-466406b2df4c" containerName="istio-proxy" containerID="cri-o://5d1c538fa1f8436b6a737c2c58c1a6b024795c4cb8c5edb43ad6c90f6be98ac1" gracePeriod=30 Jul 29 01:30:49.005393 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:49.005366 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:30:49.010177 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:49.010152 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 01:30:49.018829 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:49.018813 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:30:49.023185 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:49.023169 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mtt7s_e60adc84-fa8b-4053-9334-fc85a835ccf7/ovn-acl-logging/0.log" Jul 29 01:30:50.336985 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.336959 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:30:50.486899 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.486822 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn764\" (UniqueName: \"kubernetes.io/projected/73708d90-9fcf-4be7-a22b-466406b2df4c-kube-api-access-bn764\") pod \"73708d90-9fcf-4be7-a22b-466406b2df4c\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " Jul 29 01:30:50.486899 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.486863 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-credential-socket\") pod \"73708d90-9fcf-4be7-a22b-466406b2df4c\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " Jul 29 01:30:50.486899 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.486885 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-workload-certs\") pod \"73708d90-9fcf-4be7-a22b-466406b2df4c\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " Jul 29 01:30:50.487131 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.486948 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73708d90-9fcf-4be7-a22b-466406b2df4c-istiod-ca-cert\") pod \"73708d90-9fcf-4be7-a22b-466406b2df4c\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " Jul 29 01:30:50.487131 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.486982 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-podinfo\") pod \"73708d90-9fcf-4be7-a22b-466406b2df4c\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " Jul 29 01:30:50.487131 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.487008 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-token\") pod \"73708d90-9fcf-4be7-a22b-466406b2df4c\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " Jul 29 01:30:50.487131 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.487035 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-data\") pod \"73708d90-9fcf-4be7-a22b-466406b2df4c\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " Jul 29 01:30:50.487131 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.487073 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-workload-socket\") pod \"73708d90-9fcf-4be7-a22b-466406b2df4c\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " Jul 29 01:30:50.487131 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.487107 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-envoy\") pod \"73708d90-9fcf-4be7-a22b-466406b2df4c\" (UID: \"73708d90-9fcf-4be7-a22b-466406b2df4c\") " Jul 29 01:30:50.487131 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.487121 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "73708d90-9fcf-4be7-a22b-466406b2df4c" (UID: "73708d90-9fcf-4be7-a22b-466406b2df4c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:30:50.487575 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.487374 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "73708d90-9fcf-4be7-a22b-466406b2df4c" (UID: "73708d90-9fcf-4be7-a22b-466406b2df4c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:30:50.487575 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.487399 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "73708d90-9fcf-4be7-a22b-466406b2df4c" (UID: "73708d90-9fcf-4be7-a22b-466406b2df4c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:30:50.487575 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.487423 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-credential-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:50.487575 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.487492 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73708d90-9fcf-4be7-a22b-466406b2df4c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "73708d90-9fcf-4be7-a22b-466406b2df4c" (UID: "73708d90-9fcf-4be7-a22b-466406b2df4c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:30:50.488092 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.488067 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-data" (OuterVolumeSpecName: "istio-data") pod "73708d90-9fcf-4be7-a22b-466406b2df4c" (UID: "73708d90-9fcf-4be7-a22b-466406b2df4c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:30:50.489232 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.489209 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "73708d90-9fcf-4be7-a22b-466406b2df4c" (UID: "73708d90-9fcf-4be7-a22b-466406b2df4c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 01:30:50.489432 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.489409 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "73708d90-9fcf-4be7-a22b-466406b2df4c" (UID: "73708d90-9fcf-4be7-a22b-466406b2df4c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:30:50.489575 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.489555 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73708d90-9fcf-4be7-a22b-466406b2df4c-kube-api-access-bn764" (OuterVolumeSpecName: "kube-api-access-bn764") pod "73708d90-9fcf-4be7-a22b-466406b2df4c" (UID: "73708d90-9fcf-4be7-a22b-466406b2df4c"). InnerVolumeSpecName "kube-api-access-bn764". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:30:50.489645 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.489556 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-token" (OuterVolumeSpecName: "istio-token") pod "73708d90-9fcf-4be7-a22b-466406b2df4c" (UID: "73708d90-9fcf-4be7-a22b-466406b2df4c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:30:50.588514 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.588492 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73708d90-9fcf-4be7-a22b-466406b2df4c-istiod-ca-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:50.588514 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.588514 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-podinfo\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:50.588667 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.588523 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:50.588667 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.588532 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-data\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:50.588667 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.588539 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-workload-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:50.588667 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.588547 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-istio-envoy\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:50.588667 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.588555 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn764\" (UniqueName: \"kubernetes.io/projected/73708d90-9fcf-4be7-a22b-466406b2df4c-kube-api-access-bn764\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:50.588667 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.588563 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73708d90-9fcf-4be7-a22b-466406b2df4c-workload-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:30:50.624559 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.624532 2558 generic.go:356] "Generic (PLEG): container finished" podID="73708d90-9fcf-4be7-a22b-466406b2df4c" containerID="5d1c538fa1f8436b6a737c2c58c1a6b024795c4cb8c5edb43ad6c90f6be98ac1" exitCode=0 Jul 29 01:30:50.624656 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.624582 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" Jul 29 01:30:50.624656 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.624597 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" event={"ID":"73708d90-9fcf-4be7-a22b-466406b2df4c","Type":"ContainerDied","Data":"5d1c538fa1f8436b6a737c2c58c1a6b024795c4cb8c5edb43ad6c90f6be98ac1"} Jul 29 01:30:50.624734 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.624662 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd" event={"ID":"73708d90-9fcf-4be7-a22b-466406b2df4c","Type":"ContainerDied","Data":"f218627653f128f1766ce1c78638f55407c5946230a2cd7368515ede6edf2eca"} Jul 29 01:30:50.624734 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.624683 2558 scope.go:117] "RemoveContainer" containerID="5d1c538fa1f8436b6a737c2c58c1a6b024795c4cb8c5edb43ad6c90f6be98ac1" Jul 29 01:30:50.633503 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.633477 2558 scope.go:117] "RemoveContainer" containerID="c46b0b746466d3f638d6b192b672ec1c69bdc264896cd5a1226c61b1ab1fa3b5" Jul 29 01:30:50.641045 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.641028 2558 scope.go:117] "RemoveContainer" containerID="5d1c538fa1f8436b6a737c2c58c1a6b024795c4cb8c5edb43ad6c90f6be98ac1" Jul 29 01:30:50.641262 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:30:50.641244 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d1c538fa1f8436b6a737c2c58c1a6b024795c4cb8c5edb43ad6c90f6be98ac1\": container with ID starting with 5d1c538fa1f8436b6a737c2c58c1a6b024795c4cb8c5edb43ad6c90f6be98ac1 not found: ID does not exist" containerID="5d1c538fa1f8436b6a737c2c58c1a6b024795c4cb8c5edb43ad6c90f6be98ac1" Jul 29 01:30:50.641334 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.641270 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d1c538fa1f8436b6a737c2c58c1a6b024795c4cb8c5edb43ad6c90f6be98ac1"} err="failed to get container status \"5d1c538fa1f8436b6a737c2c58c1a6b024795c4cb8c5edb43ad6c90f6be98ac1\": rpc error: code = NotFound desc = could not find container \"5d1c538fa1f8436b6a737c2c58c1a6b024795c4cb8c5edb43ad6c90f6be98ac1\": container with ID starting with 5d1c538fa1f8436b6a737c2c58c1a6b024795c4cb8c5edb43ad6c90f6be98ac1 not found: ID does not exist" Jul 29 01:30:50.641334 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.641287 2558 scope.go:117] "RemoveContainer" containerID="c46b0b746466d3f638d6b192b672ec1c69bdc264896cd5a1226c61b1ab1fa3b5" Jul 29 01:30:50.641567 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:30:50.641550 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c46b0b746466d3f638d6b192b672ec1c69bdc264896cd5a1226c61b1ab1fa3b5\": container with ID starting with c46b0b746466d3f638d6b192b672ec1c69bdc264896cd5a1226c61b1ab1fa3b5 not found: ID does not exist" containerID="c46b0b746466d3f638d6b192b672ec1c69bdc264896cd5a1226c61b1ab1fa3b5" Jul 29 01:30:50.641613 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.641571 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c46b0b746466d3f638d6b192b672ec1c69bdc264896cd5a1226c61b1ab1fa3b5"} err="failed to get container status \"c46b0b746466d3f638d6b192b672ec1c69bdc264896cd5a1226c61b1ab1fa3b5\": rpc error: code = NotFound desc = could not find container \"c46b0b746466d3f638d6b192b672ec1c69bdc264896cd5a1226c61b1ab1fa3b5\": container with ID starting with c46b0b746466d3f638d6b192b672ec1c69bdc264896cd5a1226c61b1ab1fa3b5 not found: ID does not exist" Jul 29 01:30:50.647723 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.647700 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd"] Jul 29 01:30:50.653767 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.653746 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-ebcf3a-openshift-default-687c57b444-lqstd"] Jul 29 01:30:50.790790 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:30:50.790709 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73708d90-9fcf-4be7-a22b-466406b2df4c" path="/var/lib/kubelet/pods/73708d90-9fcf-4be7-a22b-466406b2df4c/volumes" Jul 29 01:31:02.815952 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:02.815921 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj"] Jul 29 01:31:02.816514 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:02.816194 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" podUID="593c24cf-d06a-4a8c-9637-66ef311d4fd0" containerName="istio-proxy" containerID="cri-o://57333d75bc878c3ddd7283fed702200878ae1c03f1caa1389a891cb0f805cc00" gracePeriod=30 Jul 29 01:31:08.070863 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.070835 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:31:08.142129 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.142046 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-envoy\") pod \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " Jul 29 01:31:08.142129 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.142093 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb7rv\" (UniqueName: \"kubernetes.io/projected/593c24cf-d06a-4a8c-9637-66ef311d4fd0-kube-api-access-jb7rv\") pod \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " Jul 29 01:31:08.142129 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.142117 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-workload-certs\") pod \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " Jul 29 01:31:08.142464 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.142138 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-token\") pod \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " Jul 29 01:31:08.142464 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.142236 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-workload-socket\") pod \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " Jul 29 01:31:08.142464 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.142263 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-podinfo\") pod \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " Jul 29 01:31:08.142464 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.142287 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istiod-ca-cert\") pod \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " Jul 29 01:31:08.142464 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.142338 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-credential-socket\") pod \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " Jul 29 01:31:08.142464 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.142376 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-data\") pod \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\" (UID: \"593c24cf-d06a-4a8c-9637-66ef311d4fd0\") " Jul 29 01:31:08.142464 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.142418 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "593c24cf-d06a-4a8c-9637-66ef311d4fd0" (UID: "593c24cf-d06a-4a8c-9637-66ef311d4fd0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:31:08.142832 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.142654 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "593c24cf-d06a-4a8c-9637-66ef311d4fd0" (UID: "593c24cf-d06a-4a8c-9637-66ef311d4fd0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:31:08.142832 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.142670 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-workload-certs\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:31:08.142832 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.142687 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "593c24cf-d06a-4a8c-9637-66ef311d4fd0" (UID: "593c24cf-d06a-4a8c-9637-66ef311d4fd0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:31:08.142832 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.142725 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "593c24cf-d06a-4a8c-9637-66ef311d4fd0" (UID: "593c24cf-d06a-4a8c-9637-66ef311d4fd0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 01:31:08.143410 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.143390 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-data" (OuterVolumeSpecName: "istio-data") pod "593c24cf-d06a-4a8c-9637-66ef311d4fd0" (UID: "593c24cf-d06a-4a8c-9637-66ef311d4fd0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:31:08.144369 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.144338 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/593c24cf-d06a-4a8c-9637-66ef311d4fd0-kube-api-access-jb7rv" (OuterVolumeSpecName: "kube-api-access-jb7rv") pod "593c24cf-d06a-4a8c-9637-66ef311d4fd0" (UID: "593c24cf-d06a-4a8c-9637-66ef311d4fd0"). InnerVolumeSpecName "kube-api-access-jb7rv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:31:08.144472 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.144381 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-token" (OuterVolumeSpecName: "istio-token") pod "593c24cf-d06a-4a8c-9637-66ef311d4fd0" (UID: "593c24cf-d06a-4a8c-9637-66ef311d4fd0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 01:31:08.144729 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.144706 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "593c24cf-d06a-4a8c-9637-66ef311d4fd0" (UID: "593c24cf-d06a-4a8c-9637-66ef311d4fd0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 01:31:08.144729 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.144714 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "593c24cf-d06a-4a8c-9637-66ef311d4fd0" (UID: "593c24cf-d06a-4a8c-9637-66ef311d4fd0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 01:31:08.243967 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.243937 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-data\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:31:08.243967 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.243962 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-envoy\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:31:08.244138 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.243971 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb7rv\" (UniqueName: \"kubernetes.io/projected/593c24cf-d06a-4a8c-9637-66ef311d4fd0-kube-api-access-jb7rv\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:31:08.244138 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.243982 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-token\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:31:08.244138 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.243990 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-workload-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:31:08.244138 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.243998 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istio-podinfo\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:31:08.244138 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.244006 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/593c24cf-d06a-4a8c-9637-66ef311d4fd0-istiod-ca-cert\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:31:08.244138 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.244014 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/593c24cf-d06a-4a8c-9637-66ef311d4fd0-credential-socket\") on node \"ip-10-0-135-11.ec2.internal\" DevicePath \"\"" Jul 29 01:31:08.711136 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.711101 2558 generic.go:356] "Generic (PLEG): container finished" podID="593c24cf-d06a-4a8c-9637-66ef311d4fd0" containerID="57333d75bc878c3ddd7283fed702200878ae1c03f1caa1389a891cb0f805cc00" exitCode=0 Jul 29 01:31:08.711393 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.711157 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" Jul 29 01:31:08.711393 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.711191 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" event={"ID":"593c24cf-d06a-4a8c-9637-66ef311d4fd0","Type":"ContainerDied","Data":"57333d75bc878c3ddd7283fed702200878ae1c03f1caa1389a891cb0f805cc00"} Jul 29 01:31:08.711393 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.711248 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj" event={"ID":"593c24cf-d06a-4a8c-9637-66ef311d4fd0","Type":"ContainerDied","Data":"669b0b54ca6478d030fd1143495fd4551831e91e2e0b911114a0e8e9195176e2"} Jul 29 01:31:08.711393 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.711271 2558 scope.go:117] "RemoveContainer" containerID="57333d75bc878c3ddd7283fed702200878ae1c03f1caa1389a891cb0f805cc00" Jul 29 01:31:08.720343 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.720325 2558 scope.go:117] "RemoveContainer" containerID="6129ecd2bd53772b65d1480e6d2f7eceac0334bf254f96dab3a24763e056dcde" Jul 29 01:31:08.727974 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.727954 2558 scope.go:117] "RemoveContainer" containerID="57333d75bc878c3ddd7283fed702200878ae1c03f1caa1389a891cb0f805cc00" Jul 29 01:31:08.728219 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:31:08.728200 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57333d75bc878c3ddd7283fed702200878ae1c03f1caa1389a891cb0f805cc00\": container with ID starting with 57333d75bc878c3ddd7283fed702200878ae1c03f1caa1389a891cb0f805cc00 not found: ID does not exist" containerID="57333d75bc878c3ddd7283fed702200878ae1c03f1caa1389a891cb0f805cc00" Jul 29 01:31:08.728268 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.728226 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57333d75bc878c3ddd7283fed702200878ae1c03f1caa1389a891cb0f805cc00"} err="failed to get container status \"57333d75bc878c3ddd7283fed702200878ae1c03f1caa1389a891cb0f805cc00\": rpc error: code = NotFound desc = could not find container \"57333d75bc878c3ddd7283fed702200878ae1c03f1caa1389a891cb0f805cc00\": container with ID starting with 57333d75bc878c3ddd7283fed702200878ae1c03f1caa1389a891cb0f805cc00 not found: ID does not exist" Jul 29 01:31:08.728268 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.728243 2558 scope.go:117] "RemoveContainer" containerID="6129ecd2bd53772b65d1480e6d2f7eceac0334bf254f96dab3a24763e056dcde" Jul 29 01:31:08.728483 ip-10-0-135-11 kubenswrapper[2558]: E0729 01:31:08.728465 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6129ecd2bd53772b65d1480e6d2f7eceac0334bf254f96dab3a24763e056dcde\": container with ID starting with 6129ecd2bd53772b65d1480e6d2f7eceac0334bf254f96dab3a24763e056dcde not found: ID does not exist" containerID="6129ecd2bd53772b65d1480e6d2f7eceac0334bf254f96dab3a24763e056dcde" Jul 29 01:31:08.728532 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.728489 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6129ecd2bd53772b65d1480e6d2f7eceac0334bf254f96dab3a24763e056dcde"} err="failed to get container status \"6129ecd2bd53772b65d1480e6d2f7eceac0334bf254f96dab3a24763e056dcde\": rpc error: code = NotFound desc = could not find container \"6129ecd2bd53772b65d1480e6d2f7eceac0334bf254f96dab3a24763e056dcde\": container with ID starting with 6129ecd2bd53772b65d1480e6d2f7eceac0334bf254f96dab3a24763e056dcde not found: ID does not exist" Jul 29 01:31:08.730857 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.730830 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj"] Jul 29 01:31:08.733090 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.733071 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-afd6d5-openshift-default-5f866cd9fb-zrmgj"] Jul 29 01:31:08.789404 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.789376 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="593c24cf-d06a-4a8c-9637-66ef311d4fd0" path="/var/lib/kubelet/pods/593c24cf-d06a-4a8c-9637-66ef311d4fd0/volumes" Jul 29 01:31:08.869897 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.869878 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-h284l_48d1fcf8-0b6f-4f98-958b-6cf70fb8939b/manager/0.log" Jul 29 01:31:08.990006 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:08.989946 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-64ss9_3f255234-9a38-4882-bd6b-95e2ccc46eb8/manager/0.log" Jul 29 01:31:09.749978 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:09.749948 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj_07ca9735-533c-46dc-be20-7decdb7be34a/manager/0.log" Jul 29 01:31:09.762942 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:09.762917 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj_07ca9735-533c-46dc-be20-7decdb7be34a/copy-manifests/0.log" Jul 29 01:31:10.105443 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:10.105409 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5f54d766bf-d4dr8_02c5f00e-77b7-4f78-982f-655bcdd3e46d/maas-api/0.log" Jul 29 01:31:10.583385 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:10.583353 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-k8868_f1cb9de5-a60c-4681-b089-f859b6c6b0ee/postgres/0.log" Jul 29 01:31:13.567050 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:13.566973 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-l5dkm_fb6b0154-70f0-474e-9bb5-b6f216183ba7/istio-proxy/1.log" Jul 29 01:31:13.681682 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:13.681655 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tdjm8_6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9/payload-pre-processing/0.log" Jul 29 01:31:13.908088 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:13.908003 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5479f5c76c-7vtwf_34df441f-d815-4b61-b519-02120ab837ea/router/0.log" Jul 29 01:31:14.359135 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:14.359106 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-6hvlz_3994aaa8-24aa-4127-b4b2-73e0a604fac3/main/0.log" Jul 29 01:31:14.585137 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:14.585107 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4bgb_9d224b23-8a91-416f-a013-1262fa2eaeeb/main/0.log" Jul 29 01:31:23.168687 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:23.168652 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jghtb_a50c1131-277e-4dd2-9fdd-dfd5d5aabdee/global-pull-secret-syncer/0.log" Jul 29 01:31:23.293340 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:23.293293 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cjfg6_703966f4-444e-4b3b-854d-3a7b67259320/konnectivity-agent/0.log" Jul 29 01:31:23.375577 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:23.375552 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-11.ec2.internal_fe0818b1e6c45ae97ec4ef99e813669f/haproxy/0.log" Jul 29 01:31:31.049059 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:31.048985 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-xdvdm_f3e33561-3dc0-4538-acba-55fcb83b3c04/cluster-monitoring-operator/0.log" Jul 29 01:31:31.383088 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:31.383053 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w45x4_52e27182-467d-47b0-992a-7e86173ab584/node-exporter/0.log" Jul 29 01:31:31.405035 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:31.405014 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w45x4_52e27182-467d-47b0-992a-7e86173ab584/kube-rbac-proxy/0.log" Jul 29 01:31:31.425817 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:31.425797 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w45x4_52e27182-467d-47b0-992a-7e86173ab584/init-textfile/0.log" Jul 29 01:31:33.106203 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.106175 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-rzc9z_a6663940-3227-4b6c-9569-69678c713609/networking-console-plugin/0.log" Jul 29 01:31:33.589727 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.589695 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs"] Jul 29 01:31:33.590049 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590037 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5905104d-ac22-4352-a818-481eb366b144" containerName="istio-proxy" Jul 29 01:31:33.590095 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590052 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5905104d-ac22-4352-a818-481eb366b144" containerName="istio-proxy" Jul 29 01:31:33.590095 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590059 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9b82bc0-df6e-4971-a109-8b98f2f2d21f" containerName="payload-pre-processing" Jul 29 01:31:33.590095 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590065 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9b82bc0-df6e-4971-a109-8b98f2f2d21f" containerName="payload-pre-processing" Jul 29 01:31:33.590095 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590076 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="080cefac-510b-4b69-841d-94bad36a16dc" containerName="payload-processing" Jul 29 01:31:33.590095 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590083 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="080cefac-510b-4b69-841d-94bad36a16dc" containerName="payload-processing" Jul 29 01:31:33.590095 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590091 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73708d90-9fcf-4be7-a22b-466406b2df4c" containerName="istio-proxy" Jul 29 01:31:33.590095 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590096 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="73708d90-9fcf-4be7-a22b-466406b2df4c" containerName="istio-proxy" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590103 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="593c24cf-d06a-4a8c-9637-66ef311d4fd0" containerName="istio-proxy" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590109 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="593c24cf-d06a-4a8c-9637-66ef311d4fd0" containerName="istio-proxy" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590119 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f127472-abcc-4857-8fde-8f1b88006a3f" containerName="payload-pre-processing" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590124 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f127472-abcc-4857-8fde-8f1b88006a3f" containerName="payload-pre-processing" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590130 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="593c24cf-d06a-4a8c-9637-66ef311d4fd0" containerName="istio-proxy" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590135 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="593c24cf-d06a-4a8c-9637-66ef311d4fd0" containerName="istio-proxy" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590152 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5905104d-ac22-4352-a818-481eb366b144" containerName="istio-proxy" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590157 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5905104d-ac22-4352-a818-481eb366b144" containerName="istio-proxy" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590163 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="803fb982-dd5b-4e9a-b7a4-48fc0bbd1290" containerName="payload-processing" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590168 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="803fb982-dd5b-4e9a-b7a4-48fc0bbd1290" containerName="payload-processing" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590224 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="593c24cf-d06a-4a8c-9637-66ef311d4fd0" containerName="istio-proxy" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590230 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9b82bc0-df6e-4971-a109-8b98f2f2d21f" containerName="payload-pre-processing" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590238 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5905104d-ac22-4352-a818-481eb366b144" containerName="istio-proxy" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590244 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="73708d90-9fcf-4be7-a22b-466406b2df4c" containerName="istio-proxy" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590251 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="73708d90-9fcf-4be7-a22b-466406b2df4c" containerName="istio-proxy" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590256 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="803fb982-dd5b-4e9a-b7a4-48fc0bbd1290" containerName="payload-processing" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590260 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="080cefac-510b-4b69-841d-94bad36a16dc" containerName="payload-processing" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590265 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="593c24cf-d06a-4a8c-9637-66ef311d4fd0" containerName="istio-proxy" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590271 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f127472-abcc-4857-8fde-8f1b88006a3f" containerName="payload-pre-processing" Jul 29 01:31:33.590317 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.590276 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5905104d-ac22-4352-a818-481eb366b144" containerName="istio-proxy" Jul 29 01:31:33.593154 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.593138 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.595293 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.595273 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-99rkt\"/\"kube-root-ca.crt\"" Jul 29 01:31:33.595293 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.595284 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-99rkt\"/\"openshift-service-ca.crt\"" Jul 29 01:31:33.595602 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.595588 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-99rkt\"/\"default-dockercfg-fls7h\"" Jul 29 01:31:33.602698 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.602650 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs"] Jul 29 01:31:33.663759 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.663733 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztslg\" (UniqueName: \"kubernetes.io/projected/217ea893-d477-441d-9312-979102e26458-kube-api-access-ztslg\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.663867 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.663770 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/217ea893-d477-441d-9312-979102e26458-podres\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.663867 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.663796 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/217ea893-d477-441d-9312-979102e26458-proc\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.663867 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.663844 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/217ea893-d477-441d-9312-979102e26458-lib-modules\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.664014 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.663885 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/217ea893-d477-441d-9312-979102e26458-sys\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.757225 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.757203 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/1.log" Jul 29 01:31:33.763639 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.763619 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-mf9ww_d4c187d4-ebdd-4a5c-9d5b-a6774464084d/console-operator/2.log" Jul 29 01:31:33.764363 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.764343 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztslg\" (UniqueName: \"kubernetes.io/projected/217ea893-d477-441d-9312-979102e26458-kube-api-access-ztslg\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.764411 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.764382 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/217ea893-d477-441d-9312-979102e26458-podres\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.764447 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.764420 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/217ea893-d477-441d-9312-979102e26458-proc\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.764481 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.764448 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/217ea893-d477-441d-9312-979102e26458-lib-modules\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.764519 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.764498 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/217ea893-d477-441d-9312-979102e26458-proc\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.764556 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.764514 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/217ea893-d477-441d-9312-979102e26458-sys\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.764556 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.764537 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/217ea893-d477-441d-9312-979102e26458-podres\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.764620 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.764583 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/217ea893-d477-441d-9312-979102e26458-lib-modules\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.764654 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.764622 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/217ea893-d477-441d-9312-979102e26458-sys\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.772515 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.772492 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztslg\" (UniqueName: \"kubernetes.io/projected/217ea893-d477-441d-9312-979102e26458-kube-api-access-ztslg\") pod \"perf-node-gather-daemonset-ssfgs\" (UID: \"217ea893-d477-441d-9312-979102e26458\") " pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:33.903501 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:33.903428 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:34.032864 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:34.032835 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs"] Jul 29 01:31:34.035662 ip-10-0-135-11 kubenswrapper[2558]: W0729 01:31:34.035629 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod217ea893_d477_441d_9312_979102e26458.slice/crio-123e1d592148ce2a948bdab8357b141628d3099105df67f9cad74b2c3051fc2b WatchSource:0}: Error finding container 123e1d592148ce2a948bdab8357b141628d3099105df67f9cad74b2c3051fc2b: Status 404 returned error can't find the container with id 123e1d592148ce2a948bdab8357b141628d3099105df67f9cad74b2c3051fc2b Jul 29 01:31:34.347455 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:34.347425 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-tnffv_0e291f3f-222d-49da-8d49-f6ca675b45c2/download-server/0.log" Jul 29 01:31:34.841254 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:34.841209 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" event={"ID":"217ea893-d477-441d-9312-979102e26458","Type":"ContainerStarted","Data":"e6669c36155dabcdd21d8760d609b902724af84b46169b41b8486f4ea136e577"} Jul 29 01:31:34.841488 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:34.841271 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:34.841488 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:34.841293 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" event={"ID":"217ea893-d477-441d-9312-979102e26458","Type":"ContainerStarted","Data":"123e1d592148ce2a948bdab8357b141628d3099105df67f9cad74b2c3051fc2b"} Jul 29 01:31:34.858119 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:34.858076 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" podStartSLOduration=1.858062187 podStartE2EDuration="1.858062187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:31:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:31:34.855835858 +0000 UTC m=+3346.515061751" watchObservedRunningTime="2026-07-29 01:31:34.858062187 +0000 UTC m=+3346.517288079" Jul 29 01:31:34.943593 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:34.943569 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-t27cm_514294bb-9bd1-4b6a-975f-f65446febeb1/volume-data-source-validator/0.log" Jul 29 01:31:36.042740 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:36.042712 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hzj7z_93783ee9-0737-44f5-a9e3-51438955f859/dns/0.log" Jul 29 01:31:36.067615 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:36.067591 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hzj7z_93783ee9-0737-44f5-a9e3-51438955f859/kube-rbac-proxy/0.log" Jul 29 01:31:36.196165 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:36.196141 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vpqgd_20137ae0-172b-4165-9f8f-87bb7df1fd80/dns-node-resolver/0.log" Jul 29 01:31:36.898355 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:36.898331 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dffdw_b5888d04-a838-4a13-ac06-5ef697b5ee59/node-ca/0.log" Jul 29 01:31:38.034219 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:38.034187 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-l5dkm_fb6b0154-70f0-474e-9bb5-b6f216183ba7/istio-proxy/0.log" Jul 29 01:31:38.061681 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:38.061656 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-l5dkm_fb6b0154-70f0-474e-9bb5-b6f216183ba7/istio-proxy/1.log" Jul 29 01:31:38.090787 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:38.090760 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tdjm8_6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9/payload-pre-processing/0.log" Jul 29 01:31:38.145254 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:38.145228 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5479f5c76c-7vtwf_34df441f-d815-4b61-b519-02120ab837ea/router/0.log" Jul 29 01:31:39.001677 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:39.001647 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6g74f_1939d4b2-0a14-4f02-aa47-98e04683a8dd/serve-healthcheck-canary/0.log" Jul 29 01:31:39.496222 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:39.496192 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-lsjv6_9da12546-7d93-4a97-a05f-0a7c83e3f782/insights-operator/0.log" Jul 29 01:31:39.635852 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:39.635828 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wtq82_a095e234-21c3-4383-ad01-d399215129a0/kube-rbac-proxy/0.log" Jul 29 01:31:39.654793 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:39.654769 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wtq82_a095e234-21c3-4383-ad01-d399215129a0/exporter/0.log" Jul 29 01:31:39.675235 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:39.675215 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wtq82_a095e234-21c3-4383-ad01-d399215129a0/extractor/0.log" Jul 29 01:31:40.856000 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:40.855974 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-99rkt/perf-node-gather-daemonset-ssfgs" Jul 29 01:31:41.755591 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:41.755563 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-h284l_48d1fcf8-0b6f-4f98-958b-6cf70fb8939b/manager/0.log" Jul 29 01:31:41.968042 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:41.968011 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-64ss9_3f255234-9a38-4882-bd6b-95e2ccc46eb8/manager/0.log" Jul 29 01:31:42.477244 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:42.477218 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj_07ca9735-533c-46dc-be20-7decdb7be34a/manager/0.log" Jul 29 01:31:42.497410 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:42.497340 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cf6bc8b6f-whwvj_07ca9735-533c-46dc-be20-7decdb7be34a/copy-manifests/0.log" Jul 29 01:31:43.890352 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:43.890320 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-896f4944f-597sf_f2bac32b-55cf-493d-94a8-6dd0c637d1e6/manager/0.log" Jul 29 01:31:44.447929 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:44.447896 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-l5dkm_fb6b0154-70f0-474e-9bb5-b6f216183ba7/istio-proxy/0.log" Jul 29 01:31:44.474963 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:44.474942 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-l5dkm_fb6b0154-70f0-474e-9bb5-b6f216183ba7/istio-proxy/1.log" Jul 29 01:31:44.494678 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:44.494654 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tdjm8_6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9/payload-pre-processing/0.log" Jul 29 01:31:44.539986 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:44.539917 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5479f5c76c-7vtwf_34df441f-d815-4b61-b519-02120ab837ea/router/0.log" Jul 29 01:31:46.183719 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:46.183687 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-l5dkm_fb6b0154-70f0-474e-9bb5-b6f216183ba7/istio-proxy/0.log" Jul 29 01:31:46.210833 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:46.210803 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-l5dkm_fb6b0154-70f0-474e-9bb5-b6f216183ba7/istio-proxy/1.log" Jul 29 01:31:46.234375 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:46.234348 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tdjm8_6b23a01d-67e9-4c9e-b88d-deecdf3bf1d9/payload-pre-processing/0.log" Jul 29 01:31:46.277436 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:46.277414 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5479f5c76c-7vtwf_34df441f-d815-4b61-b519-02120ab837ea/router/0.log" Jul 29 01:31:48.887446 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:48.887414 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-m7dmb_dd5b2b10-6978-4787-9bec-74219be83e1f/migrator/0.log" Jul 29 01:31:48.907521 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:48.907498 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-m7dmb_dd5b2b10-6978-4787-9bec-74219be83e1f/graceful-termination/0.log" Jul 29 01:31:49.247544 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:49.247520 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb_ea6878e6-f9c1-4f52-8387-2c2c38d7084f/kube-storage-version-migrator-operator/1.log" Jul 29 01:31:49.248439 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:49.248415 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-95nkb_ea6878e6-f9c1-4f52-8387-2c2c38d7084f/kube-storage-version-migrator-operator/0.log" Jul 29 01:31:50.500067 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:50.500037 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c527k_cb151e18-2bf5-4098-a956-829f7bbf9e4e/kube-multus-additional-cni-plugins/0.log" Jul 29 01:31:50.519883 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:50.519858 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c527k_cb151e18-2bf5-4098-a956-829f7bbf9e4e/egress-router-binary-copy/0.log" Jul 29 01:31:50.539605 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:50.539580 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c527k_cb151e18-2bf5-4098-a956-829f7bbf9e4e/cni-plugins/0.log" Jul 29 01:31:50.560435 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:50.560413 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c527k_cb151e18-2bf5-4098-a956-829f7bbf9e4e/bond-cni-plugin/0.log" Jul 29 01:31:50.580125 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:50.580103 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c527k_cb151e18-2bf5-4098-a956-829f7bbf9e4e/routeoverride-cni/0.log" Jul 29 01:31:50.600200 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:50.600184 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c527k_cb151e18-2bf5-4098-a956-829f7bbf9e4e/whereabouts-cni-bincopy/0.log" Jul 29 01:31:50.621221 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:50.621203 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c527k_cb151e18-2bf5-4098-a956-829f7bbf9e4e/whereabouts-cni/0.log" Jul 29 01:31:50.817733 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:50.817704 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-w8sxh_41c070b5-13d1-47ea-a5e4-50f519e7e647/kube-multus/0.log" Jul 29 01:31:50.877325 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:50.877278 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mgvr7_a00c6eb4-e6cb-4486-949f-f4f4a7ba538c/network-metrics-daemon/0.log" Jul 29 01:31:50.895265 ip-10-0-135-11 kubenswrapper[2558]: I0729 01:31:50.895239 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mgvr7_a00c6eb4-e6cb-4486-949f-f4f4a7ba538c/kube-rbac-proxy/0.log"